NAVAL  POSTGRADUATE  SCHOOL 

Monterey,  California 


THESIS 


EVALUATION  OF  USER  INFORMATION  SATISFACTION  OF 

THE  AUTOMATED  QUALITY  OF  CARE  EVALUATION 

SUPPORT  SYSTEM 

by 

James  Runyan  Booth 

and 

John  Linwood  Bryant,  Jr. 

September,  1991 


Thesis  Advisor: 


Moshe  Zviran 


Approved  for  public  release;  distribution  is  unlimited 


T253896 


Unclassified 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


REPORT  DOCUMENTATION  PAGE 


la  REPORT  SECURITY  CLASSIFICATION 
UNCLASSIFIED 


1b  RESTRICTIVE  MARKINGS 


2a  SECURITY  CLASSIFICATION  AUTHORITY 


2b  DECLASSIFICATION/DOWNGRADING  SCHEDULE 


3   DISTRIBUTION/AVAILABILITY  OF  REPORT 
Approved  for  public  release;  distribution  is  unlimited. 


4  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 


5  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 


6a  NAME  OF  PERFORMING  ORGANIZATION 
Naval  Postgraduate  School 


6b  OFFICE  SYMBOL 
(If  applicable) 
55 


7a  NAME  OF  MONITORING  ORGANIZATION 
Naval  Postgraduate  School 


6c  ADDRESS  (C/ty,  State,  and  ZIP  Code) 
Monterey,  CA  93943  5000 


7b  ADDRESS  (City,  State,  and  ZIP  Code) 
Monterey,  CA  93943-5000 


8a  NAME  OF  FUNDING/SPONSORING 
ORGANIZATION 


8b  OFFICE  SYMBOL 
(If  applicable) 


9  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 


8c  ADDRESS  (City,  State,  and  ZIP  Code) 


10  SOURCE  OF  FUNDING  NUMBERS 


Program  tlemenl  No 


Proiea  No 


Work  unit  A(.<.eiiion 
Number 


1 1  TITLE  (Include  Security  Classification) 

EVALUATION  OF  USER  INFORMATION  SATISFACTION  OF  THE  AUTOMATED  QUALITY  OF  CARE  EVALUATION  SUPPORT 

SYSTEM  (UNCLASSIFIED) 


12  PERSONAL  AUTHOR(S)    Booth,  James  R.  and  Bryant,  John  L.  Jr. 


13a  TYPE  OF  REPORT 
Master's  Thesis 


13b  TIME  COVERED 
From  To 


14  DATE  OF  REPORT  (year,  month,  day) 
September  1991 


15   PAGE  COUNT 
187 


16  SUPPLEMENTARY  NOTATION 

The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the  official  policy  or  position  of  the  Department  of  Defense  or  ■  he  U.S. 

Government. 


17  COSATI  CODES 


FIELD 


GROUP 


SUBGROUP 


18  SUBJECT  TERMS  (continue  on  reverse  if  necessary  and  identify  by  block  number) 
Information  Systems,  measurement,  satisfaction,  effectiveness,  evaluation 


1 9  ABSTRACT  (continue  on  reverse  if  necessary  and  identify  by  block  number) 

This  thesis  investigates  the  level  of  User  Information  Satisfaction  of  the  Automated  Quality  ot  Care  Evaluation  Support  System  (AQCESS) 
and  the  Composite  Health  Care  System  (CHCS).  A  User  Information  Satisfaction  questionnaire  was  administered  to  AQCESS  users  at  Silas  B. 
Hays  (Army  I  Hospital,  Fort  Ord,  California  and  the  Naval  Hospital,  Pensacola,  Florida.  The  findings  from  the  AQCESS  system  are  compared  to 
those  obtained  from  the  CHCS  system  at  the  Naval  Hospital,  Charleston,  South  Carolina. 

Significant  differences  in  satisfaction  between  work  groups  were  found.  Overall,  physicians  were  least  satisfied  and  administrative  personnel 
were  the  most  satisfied  of  the  groups. 


20  DISTRIBUTION/AVAILABILITY  OF  ABSTRACT 

Q  UNClASSIHED/UNUMITtD  J   SAMt  AS  KEPOH1         M    OTIC  USERS 


22a  NAME  OF  RESPONSIBLE  INDIVIDUAL 
Moshe  Zviran 


21   ABSTRACT  SECURITY  CLASSIFICATION 
Unclassified 


22b  TELEPHONE  (Include  Area  code) 
646  2489 


22c  OFFICE  SYMBOL 
AS/ZV 


DD  FORM  1473. 84  MAR 


83  APR  edition  may  be  used  until  exhausted 
All  other  editions  are  obsolete 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


Approved  for  public  release;  distribution  is  unlimited 


Evaluation  of  User  Information  Satisfaction 
of  the  Automated  Quality  of  Care 
Evaluation  Support  System 

by 

James  Runyan  Booth 

Lieutenant  Commander,  United  States  Navy 

B.S.,  Alabama  State  University,  1974 

and 

John  Linwood  Bryant,  Jr. 
Lieutenant,  United  States  Navy 
B.S.,  Rutgers  University,  1982 

Submitted  in  partial  fulfillment 
of  the  requirements  for  the  degree  of 

MASTER  OF  SCIENCE  IN  INFORMATION  SYSTEMS 

from  the 

NAVAL  POSTGRADUATE  SCHOOL 
SEPTEMBER  1991 

r  


ABSTRACT 


This  thesis  investigates  the  level  of  User  Information 
Satisfaction  of  the  Automated  Quality  of  Care  Evaluation  Support 
System  (AQCESS)  and  the  Composite  Health  Care  System  ( CHCS ) .  A 
User  Information  Satisfaction  questionnaire  was  administered  to 
AQCESS  users  at  Silas  B.  Hays  (Army)  Hospital,  Fort  Ord,  California 
and  the  Naval  Hospital,  Pensacola,  Florida.  The  findings  from  the 
AQCESS  system  are  compared  to  those  obtained  from  the  CHCS  system 
at  Naval  Hospital,  Charleston,  South  Carolina. 

Significant  differences  in  satisfaction  between  work  groups 
were  found.  Overall,  physicians  were  least  satisfied  and 
administrative  personnel  were  the  most  satisfied  of  the  groups. 
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INTRODUCTION 


A.   OVERVIEW 

Top  management  personnel  often  attempt  to  quantify  or 
measure  the  value  of  information  systems  technology  to  their 
organization.  Managers  within  the  Information  Systems  (IS) 
community,  in  turn,  debate  and  search  for  improved  techniques 
to  measure  and,  otherwise,  make  improvements  to  both  existing 
and  future  systems. 

In  1984  and  1986,  the  Society  for  Information  Management 
(SIM)  along  with  the  Management  Information  Systems  Research 
Center  (MISRC)  conducted  surveys  to  determine  key  information 
systems  issues  facing  IS  executives/managers  over  the  next 
five  years.  An  issue  ranked  among  the  top  ten  in  importance 
and  one  which  has  long  been  an  IS  management  problem  was 
system  effectiveness  and  its  measurement  ( Brancheau  and 
Wetherbe,  1987). 

Inasmuch  as  the  Department  Of  Defense  (DoD)  has  a  number 
of  large,  expensive  information  systems  being  used  at  various 
Medical  Treatment  Facilities,  a  simple  and  efficient  method  to 
measure  and  analyze  the  effectiveness  of  these  systems  is 
essential.  This  is  a  very  difficult  concept  to  narrow  down 
because  of  differing  opinions  or  ideas  of  value  (ie;  economic, 


efficiency,  effectiveness,  user  satisfaction,  etc.).  Many 
scholars  of  this  problem  have  determined  and  agreed  that  all 
of  these  factors  are  important  ones,  however,  there  is  a  need 
to  concentrate  on  the  effectiveness  of  a  system  in  terms  of 
user  interface.  In  other  words,  user  satisfaction  of  the 
system.  This  is  due  to  the  fact  that  no  matter  how  "good"  the 
system  may  be,  if  the  user  is  not  satisfied  with  it,  it 
becomes  worthless.  However,  if  the  user  is  satisfied,  the 
system  may  be  considered  to  be  effective,  provided  all  other 
organizational  objectives  are  met. 

B.   RESEARCH  OBJECTIVES 

This  study  is  a  follow-on  to  the  Evaluation  of  User 
Information  Satisfaction  of  the  Composite  Health  Care  System 
(CHCS)  (Hurd,  1991)  and  will  empirically  evaluate  the 
effectiveness  of  the  Automated  Quality  of  Care  Evaluation 
Support  System  (AQCESS)  and  compare  them  to  those  found  for 
CHCS.  A  standardized,  previously  validated,  satisfaction 
survey  was  administered  to  users  of  AQCESS  at  the  Naval 
Hospital,  Pensacola,  Florida  and  Silas  B.  Hayes  Hospital,  Fort 
Ord,  California  to  measure  intrinsic  satisfaction 
characteristics.  The  results  of  the  survey  will  be  analyzed 
using  relevant  statistical  methods  in  order  to  identify  and 
document  problem  areas  if  they  exist,  as  well  as  those  areas 
considered  to  be  positive  aspects  of  the  system.  Upon 
completion,  this  study  will  formulate  a  baseline  measure  of 


the  AQCESS  user's  satisfaction  at  both  sites  and  investigate 
possible  cause  and  effect  relationships.  Future  comparisons 
can  be  made  using  the  same  standard  survey  at  the  study 
hospital  or  at  other  facilities. 


II.   THE  AUTOMATED  QUALITY  OF  CARE  EVALUATION  SUPPORT  SYSTEM 


B .   BACKGROUND 

The  Tri-Service  Medical  Information  Systems  (TRIMIS) 
Program  Office  was  formally  created  in  July  1974,  by  the 
Department  of  Defense  Assistant  Secretaries  of  Defense 
(Comptroller,  Health  and  Environment) .  The  program,  which  is 
now  part  of  Defense  Medical  Systems  Support  Center  (DMSSC)  of 
the  Office  of  the  Assistant  Secretary  of  Defense  Health 
Affairs,  was  assigned  the  task  of  consolidating  previous 
service  efforts  and  to  "improve  the  effectiveness  and  economy 
of  health  care  delivery  in  the  Army,  Navy  and  Air  Force." 
TRIMIS  refined  its  mission  objectives  by  stating  that  it  would 
develop  automated  information  systems  for  timely  patient- 
centered  health  data.  These  systems  would  support  medical 
services,  clinical  research,  epidemiological  studies  and 
health  care  information. 

The  Hospital  Services  Program  Office  (HSPO)  developed  a 
computer  supported  clinical  records  and  patient  administration 
system  using  the  MUMPS  language  combined  with  certain 
utilities  from  the  Veterans  Administration  File  Manager.  The 
system  received  extensive  tri-service  input  and  was  designed 


incorporated  comprehensive  service-specific  information  to 
ensure  accurate  and  reliable  data  accumulation.  The  system 
was  also  designed  to  be  user-friendly  (easy  to  learn,  able  to 
provide  online  assistance,  and  able  to  operate  without 
dedicated  computer  operators/special  environmental 
conditions).  The  MUMPS  language  allowed  for  Hardware 
flexibility  (easily  modified  to  correct  problems  and  integrate 
additional  reguirements ) .  Development  of  this  system, 
however,  was  suspended  following  redirection  of  the  TRIMIS 
program  in  March  of  1984.  (AQCESS  User  Manual,  Issues  2.0, 
2.2,  2.4) 

B.   AQCESS  PURPOSE 

In  August  of  1984,  the  Health  Affairs  Office  of  the 
Assistant  Secretary  of  Defense  identified  medical  guality 
assurance  as  being  a  priority  within  the  military  health  care 
system.  The  TRIMIS  Office,  along  with  representatives  from 
the  Army,  Navy  and  Air  Force,  were  reguested  by  the 
Professional  Affairs  and  Quality  Assurance  Office  of  the 
Deputy  Assistant  Secretary  of  Defense  to  develop  and  implement 
a  computer  supported  information  system  at  all  DoD  hospitals 
by  1985.  The  resulting  system  was  the  Automated  Quality  of 
Care  Evaluation  Support  System  (AQCESS).  This  system,  also 
written  in  MUMPS,  was  designed  to  satisfy  the  following 
objectives:  (1)  Collect  and  report  clinical,  administrative 
and  managerial  information  necessary  to  support  inpatient 


administration  of  the  medical  quality  assurance  programs 
within  DoD;  (2)  Improve  the  quality  and  timeliness  of  health 
care  evaluation;  and  (3)  Support  the  identification  of 
variations  that  would  adversely  affect  the  quality  of  health 
care.  (Assistant  Secretary  of  Defense,  Health  Management 
Systems,  1984) 

C.   AQCESS  SYSTEM  MODULES 

AQCESS  is  an  interactive,  mini-computer  based,  online 
computer  system.   It  includes  the  following  modules: 

1.  Admission  and  Disposition 

Admission  and  Disposition  addresses  patient 
registration,  admission,  transfer,  disposition,  bed 
management,  inpatient  history  and  reporting.  The  user  inputs 
patient  information/data  onto  a  formatted  screen  and  has  the 
capability  to  manipulate  (ie;  edit,  view,  cancel,  track, 
print,  validate)  as  necessary.  From  there,  a  number  of  other 
items  may  be  generated  (ie;  register  number,  remarks  section, 
inpatient  history,  query  of  patient  database,  general/specific 
reports ) . 

2 .  Clinical  Records 

Clinical  Records  includes  documentation  on  patient 
episodes,  diagnosis  and  procedure  data  capture,  patient  day 
computation,  record  tracking  and  reporting.  The  user  can 
collect,  edit,  validate,  display,  track  various  items, 
compute/maintain  data  and  generate  reports. 


3.  Quality  Assurance 

Quality  Assurance  supports  inpatient  and  emergency 
room  occurrence  screening,  problem  tracking,  offers  solutions/ 
recommendations,  provider  profiling,  credentialling,  incident 
reporting,  problem  audit  tracking,  drug  utilization,  infection 
control,  blood  utilization  review  and  reporting,  risk 
management  and  surgical  case  review.  Additionally,  through 
monitoring  and  evaluation,  the  QA  subsystem  can  gather  data 
pertaining  to  the  MTF's  administrative  procedures.  It  is  then 
used  to  generate  reports  and  trend  analysis  with  respect  to 
the  quality  of  care. 

4.  Ad  Hoc  Reporting 

Ad  Hoc  Reporting  enables  users  to  produce  AQCESS 
reports  in  support  of  management  and  clinical  research 
studies.  It  can  produce  very  detailed  reports,  extract  data 
from  multiple  files,  compute  new  data  from  existing  data, 
enable  any  data  item  to  be  ported  to  any  deliverable  form,  use 
boolean  logic  selection,  conduct  multi-level  sorting,  perform 
computations,  provide  flexible  report  formatting,  generate 
MUMPS  code, support  system  security. 

5.  Embosser  Interface 

Embosser  Interface  supports  a  number  of  patient 
(input)  information  card  embossers.  The  embosser  produces  a 
credit  card-sized,  raised  letter  card  which  is  subsequently 


used  much  like  a  credit  card  (patient  information  obtained 
from  card  for  input  into  the  system). 

6.  Interface  to  the  Defense  Enrollment  Eligibility 
Reporting  System  (DEERS) 

Interface  to  DEERS  provides  the  capability  to  check 
patient  eligibility  at  the  time  of  admission  as  well  as  input 
changes/corrections  to  both  the  AQCESS  and  DEERS  databases. 

7.  Business  Office 

Business  Office  includes  cashier  processing  and  a 
number  of  other  accounting  and  general  business/office 
functions.  It  enables  users  to  calculate  inpatient/outpatient 
bills,  review  schedule  of  accounts,  post  accounts  received, 
display  and  update  the  rate  schedule  summary  and  issue 
receipts/invoices . 

8 .  Outpatient  Encounter 

Outpatient  Encounter  supports  outpatient  clinics  and 
provides  them  with  the  capability  to  schedule,  cancel  or  scan 
for  appointments  ( patient  appointments  that  are  pending  and  to 
search  for  available  slots),  process  a  waiting  list  request, 
create/edit  a  schedule,  check  in  patients  for  appointments  and 
enter  registration  data  for  new  patients. 

9.  Emergency  Room 

Emergency  Room  allows  users  to  record  patient 
emergency  room  visits;  produces  an  automated  SF  Form  558 
( Emergency  Treatment  Record ) ;  produces  the  automated  emergency 
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room  log;  and  stores,  online,  a  history  of  the  encounter.  The 
Emergency  Room  module  is  no  longer  used  because  of  reduced 
administrative  manpower  ( system  seems  to  be  too  time  consuming 
under  ER  conditions). 

D .   OPERATION 

AQCESS  utilizes  a  centralized  database  and  displays 
data/information  by  means  of  a  series  of  screens.  It  provides 
real-time  information  processing  and  generates  several 
automated  outputs  on  a  daily  and  monthly  basis.  The  terminal 
displays  a  sign-on  screen  on  which  the  user  enters  a  user  ID 
and  password.  After  the  system  validates  this  input,  the 
AQCESS  main  menu  is  displayed.  (If  the  user  enters  an  invalid 
ID  and/or  pass-word  more  than  a  predetermined  number  of  times, 
"User  Entry"  will  lock  the  terminal  and  user  ID. )  The  menu 
indicates  the  functions  authorized  to  that  user  along  with  the 
specific  functions  which  can  be  performed  at  that  particular 
terminal.  From  the  main  menu,  AQCESS  data/information  is 
subsequently  displayed  through  a  series  of  approximately  100 
screens  associated  with  each  sub-menu/  selection.  The  number 
of  different  responses  users  can  make  to  each  of  the  displays 
of  data  entered  for  each  patient  record  (must  consider  a 
number  of  unique  combinations)  can  be  estimated  as  being 
infinite.  There  are  a  number  of  recovery  and  error  correction 
procedures.  However,  software  errors  which  cause  the  user  to 
be  deleted  from  the  system  are  only  corrected  by  the  Systems 


corrected  by  the  Systems  Manager  or  technical  assistants.  If 
a  user  is  authorized  to  access  data  via  a  systems  function,  in 
most  cases  he/she  is  also  authorized  to  update  the  data 
available  through  that  function  at  any  time.  An  error  will 
result  when  a  user  (e.g., in  admissions)  attempts  to  update 
certain  types  of  data  elements  (e.g., the  source  of  admission) 
because  these  data  elements  can  only  be  updated/changed  by  the 
Systems/Corrections  Manager. 

As  patients  are  registered  and  admitted,  the  user  inputs 
demographic  data  and  the  DEERS  eligibility  check  is  performed. 
An  embossed  card  is  issued,  if  requested,  and  it  contains 
input  information  (register/account  number,  patient  name, 
family  member  prefix,  sponsor  SSN,  date  of  birth,  sex)  for 
access  to  patient's  record.  Upon  patient's  release  from  the 
MTF,  further  data  is  entered  by  the  dispositions  clerk, 
cashiering  clerk  and  clinical  records  office.  The  database 
can  then  provide  screen  or  hardcopy  reports  on  current 
listings  of  patients  admitted  and  discharged,  current 
inpatients,  patients  on  convalescent  leave,  patients 
subsisting  elsewhere,  patients  on  medical  hold  as  well  as 
listings  of  inpatient  or  delinquent  bills.  Once  the  patient 
has  been  admitted,  the  data  is  also  tracked  by  the  Quality 
Assurance  Coordinator  until  he/she  is  discharged. 
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1 .  System  Management 

System  Management  allows  the  System  Manager  to  modify 
system  tables  (which  define  the  valid  entries  for  specific 
data  fields) ,  maintain  hospital  profile  data,  allows  manager 
the  flexibility  to  decide  whether  to  assign  register  numbers 
automatically  or  manually,  generate  user  ID' s  and  passwords, 
assign  functional  privilege  to  users/terminals,  broadcast 
system  announcements,  disable  and  re-enable  user  logon 
capability  system-wide,  control  the  active  hours  for  the  DEERS 
line,  view  the  report  queue  and  interrupt  processing,  archive 
records,  perform  system  back-ups,  list  software  error  logs, 
load  software  updates  and  monitor  disk  space  usage. 

2 .  Security 

Because  of  the  sensitive  nature  of  the  data  collected 
and  reported  by  AQCESS,  security  of  this  data  is  of  utmost 
importance.  Briefly,  system  security  is  ensured  by  the 
following  measures: 

•  Use  of  systems  functions  and  access  to  data  displays  and 
reports  are  restricted  to  authorized  users  and  designated 
terminals,  as  well  as  password. 

•  Data  is  included  on  reports  in  the  form  of  specific  codes 
to  protect  confidentiality. 

•  The  system  includes  a  time-out  feature,  causing  the  screen 
display  to  disappear  if  a  terminal  is  left  unattended  past 
a  predetermined  time  period. 

•  A  privacy  act  statement  is  printed  as  the  banner/cover 
page  of  every  system  report.  (AQCESS  User  Manual,  Issues 
2.0,  2.2,  2.4) 
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E.   SYSTEM  BENEFITS 

Generally,   the   overall   benefits   of   AQCESS   include: 

1.  automated  data  collection 

2.  real-time  processing 

3.  easily  accessible  clinical  information 

4.  manipulation/analysis  of  data  for  easy  generation   of 
reports 

5.  improved  quality  assurance  monitoring 

6 .  improved  security  features 

7.  improved  system  management  capabilities 
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III.   DESCRIPTION  OF  THE  COMPOSITE  HEALTH  CARE  SYSTEM 


A .  BACKGROUND 

The  Department  of  Defense  (DoD)  has  pursued  the  goal  of 
providing  automated  computer  support  to  its  hospitals  and 
clinics  since  1968.  In  February  1979,  the  Mission  Elements 
Need  Statement  (MENS),  establishing  the  need  for  an  automated 
computer  system  was  approved.  Under  the  direction  of  the  Tri- 
Service  Medical  Information  System  (TRIMIS)  program  office, 
stand-alone  and  integrated  health  care  computer  systems  were 
acquired,  implemented  and  operated  to  support  Pharmacy 
(TRIPHARM),  Laboratory  (TRILAB),  Radiology  (TRIRAD), 
Appointment  and  Scheduling  (TRIPASS),  Quality  Assurance 
(AQCESS),  and  Hospital  Information  System  (HIS).  The 
knowledge  and  experience  gained  through  the  operation  of  these 
systems  since  1979  was  used  to  refine  and  validate  the  system 
requirements  for  a  completely  new  and  fully  integrated 
information  system  the  Composite  Health  Care  System  (CHCS). 

B.  CHCS  PURPOSE 

The  primary  purpose  of  CHCS  is  "to  provide  health  care 
services  that  support  military  forces  in  fulfilling  their 
required  mission...."  (Draft  System  Decision  Paper,  1989). 
Improving  the  efficiency  and  effectiveness  of  military  health 
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care  delivery  during  peacetime  and  during  mobilization  through 
integration  of  information  resources  is  the  goal  of  CHCS.  The 
major  focus  of  CHCS  is  the  integration  of  shared  information 
resources  to  resolve  the  deficiencies  of  communication, 
decision  support,  and  information  processing.  The  Composite 
Health  Care  System  is  designed  as  a  fully  integrated  medical 
information  system  that  provides  automated  support  of 
information  requirements  for  military  medical  treatment 
facilities.  CHCS  supports  the  administrative  functions  and 
the  delivery  of  health  care  with  information  retrieval 
services. 

C.  DEPLOYMENT  STATUS 

CHCS  is  undergoing  the  Operational  Test  and  Evaluation 
(OT&E)  phase  and  is  currently  deployed  to  Naval  Hospital 
Charleston  as  one  beta-test  site.  The  role  of  a  beta-test 
site  is  to  incrementally  replace  the  TRIMIS  systems  with 
validated  CHCS  software.  Continued  validation  of  requirements 
and  extensive  evaluation  of  the  CHCS  software  is  conducted 
prior  to  the  decision  to  deploy  CHCS  worldwide. 

D.  OPERATIONAL  FEATURES 

CHCS  supports  various  functional  areas  in  sharing 
information.  Each  functional  area  uses  this  information  for 
its  own  purpose  and  communicates  results/activities  to  other 
areas.   Functional  areas  supported  by  CHCS  include: 
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1.  Patient  Administration 

Patient  administration  does  the  registration  of  a 
patient  into  the  system.  Once  in  the  computer  system,  the 
patient's  demographic  information  is  available  for  access  by 
all  other  modules.  The  patient  administration  module  also 
performs  the  functions  to  admit  patients  to  the  hospital, 
transfer  patients  between  wards,  and  discharge  patients  from 
the  hospital.  The  creation,  update  and  closing  of  inpatient 
records  is  also  performed  by  the  patient  administration 
module. 

2.  Patient  Appointment  and  Scheduling 

The  Patient  Appointment  and  Scheduling  module  provides 
for  a  centralized  appointment  service,  a  decentralized 
appointment  service,  or  a  combination  of  both  depending  on  the 
desire  of  the  hospital.  The  appointment  service  creates  and 
maintains  the  appointment  schedule  for  the  hospital .  A 
patient  is  given  an  appointment  date  and  time  to  see  the 
health  care  provider  creating  a  centralized  repository  of 
information  to  produce  operational  reports  of  activity. 

3.  Nursing 

CHCS  supports  inpatient  nursing  in  a  variety  of  areas. 
In  the  area  of  patient  care,  nursing  uses  the  system  for 
entering  and  obtaining  the  status  of  all  physician  orders, 
entering  patient  assessment  data,  and  generating  patient  care 
plans.   Documentation  of  patient  progress  is  performed  in  the 
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automated  nursing  notes.  Nursing  unit  management  uses  CHCS  to 
generate  shift  care  plans,  patient  management  reports,  drug 
administration  time  reports,  and  staffing  requirement  reports. 
Routine  administrative  functions  are  performed  by  CHCS  for 
nursing  including:  staff  credentialling,  documentation  of 
continuing  education  and  in-service  training,  and  staff 
scheduling  capabilities. 

4 .  Laboratory 

The  laboratory  uses  the  CHCS  system  for  processing  of 
orders  for  laboratory  tests  with  automated  reporting  of  test 
results.  Specimens  are  processed,  tracked,  and  reported 
through  the  system.  All  results  are  placed  into  the  patients 
automated  medical  record  and  are  immediately  available  for 
inquiry  by  health  care  providers.  Automated  control  of  blood 
bank  operations  including:  blood  acquisition,  inventory  and 
utilization  are  accomplished  through  CHCS.  The  laboratory  is 
supported  with  ability  to  manage  inventory  control,  register 
patients  into  the  tumor  registry,  conduct  a  drug  testing 
program,  and  generate  a  multitude  of  management  reports. 

5 .  Pharmacy 

The  pharmacy  uses  CHCS  to  process  prescriptions  for 
patients  that  are  ordered  by  health  care  providers.  The 
health  care  provider  enters  the  prescription  information  at  a 
terminal  located  in  his  office.  The  information  is 
electronically  transmitted  to  the  pharmacy.   The  pharmacy  can 
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immediately  prepare  the  medication  for  dispensing  before  the 
patient  leaves  the  physician's  office.  CHCS  automatically 
performs  checks  for  patient  allergies,  drug  interactions  with 
medications  the  patient  is  currently  taking,  drug  dosage  for 
the  age  of  the  patient,  and  records  the  prescription  into  the 
patient's  medical  record.  In-patient  order  processing  is 
accomplished  in  much  the  same  fashion.  On-line  drug 
monographs  are  available  for  inguiry  by  medical  staff  at  any 
terminal .  Pharmacy  management  of  inventory  control  and 
reporting  are  also  accomplished. 

6.  Radiology 

Orders  for  radiological  procedures  are  electronically 
transmitted  to  the  radiology  department  where  the  order  is 
processed.  Order  tracking  and  results  reporting  are  performed 
by  the  system.  Radiology  results  are  immediately  available 
for  inquiry  by  medical  staff.  Department  management  including 
inventory  control  is  also  conducted  through  CHCS. 

7.  Clinical  Dietetics 

The  Clinical  Dietetics  service  uses  CHCS  to  obtain 
patient  data  needed  to  access  the  nutritional  needs  of  the 
patient.  This  information  is  used  to  create  patient  diet 
plans,  select  patients  of  interest,  patient  menu  selections 
and  monitor  patient  nutritional  data.  CHCS  is  also  used  to 
maintain  the  nutrition  clinic  schedule  and  compute  clinical 
dietetics  workload  data. 
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E.   POTENTIAL  BENEFITS 

In  general  terms,  the  overall  benefits  of  the  CHCS  system 
include: 

1.  reliable,    timely,    easily   accessible    clinical 
information 

2.  improved  documentation  for  medicolegal  issues 

3.  improved  communication 

4.  improved  patient  satisfaction 

5.  improved  management  efficiency 
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IV.   USER  INFORMATION  SATISFACTION  AND  ITS  MEASUREMENT 


A.  INTRODUCTION 

While  the  cost  of  hardware  and  telecommunications 
continues  to  decrease,  the  cost  of  software  development  and 
maintenance  for  "systems"  is  continuously  rising. 
Organizations  spend  considerable  amounts  of  time  and  money  to 
create  and  implement  useful/successful  information  systems. 
The  first  "hurdle"  to  be  cleared  is  whether  or  not  the  system 
is  really  needed.  If  the  need  is  there  and  the  system  is 
developed,  the  system's  functionality  must  be  evaluated  (Is  it 
doing  what  it's  supposed  to  do?).  Therefore,  in  addition  to 
gauging  the  system  by  cost-benefit  analysis,  the  effectiveness 
and  intangible  benefits  of  the  system  need  to  be  evaluated. 

B.  METHODS  TO  ASSESS  BENEFITS/EFFECTIVENESS 

Several  procedures  have  been  suggested  for  assessing  the 
benefits  or  effectiveness  of  an  information  system.  Possibly 
due  to  the  level  of  knowledge  possessed  by  today's  users, 
measuring  user  satisfaction  is  a  common  consideration  in  these 
procedures.  User  satisfaction  is  a  very  critical  yardstick  in 
measuring  computer  system  success  and  failure.  (Powers  and 
Dickson,  1973)   How  can  it  be  measured? 
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System  usage  (Swanson,  1974;  Conrath  and  Mignen,  1990) 
assumes  that  the  effectiveness  of  an  information  system  can  be 
evaluated  by  the  amount  of  time  the  system  is  used.  Simply, 
one  uses  an  effective  information  system  more  than  a  less 
effective  system.  The  assumption  here  is  that  the  use  of  the 
system  is  optional  to  the  user.  If,  however,  the  use  of  the 
system  is  mandatory,  the  correlation  between  use  and 
effectiveness  can  no  longer  be  made. 

Others  (e.g.,  Hamilton  and  Chervany,  1981)  contend  that 
effectiveness  is  determined  by  comparing  performance  to  pre- 
determined objectives.  To  assess  the  effectiveness  of  an 
information  system,  the  task  objective  of  the  system  is  first 
agreed  upon.  Standards  are  developed  to  measure  how  well  the 
information  system  performs  each  task.  Some  of  the  problems 
which  emerge  include:  objectives  and  measures  are  often  not 
sufficiently  defined;  easily  implemented,  efficiency-oriented 
measures  are  regularly  used  over  complex,  effectiveness- 
oriented  measures;  individual  or  conflicting  interpretations 
of  what  the  objectives  and  corresponding  data  measures  are, 
often  exist. 

Another  frequently  used  procedure  to  determine  the 
benefits/value  of  an  information  system  is  economic  evaluation 
(cost-benefit  analysis).  The  benefits  of  a  system  are  weighed 
against  the  cost  for  development  and  operation  of  the  system 
(Nolan,  1974).  Although  this  appears  objective  and 
comprehensive  on  paper,  in  practice,  cost-benefit  analysis  is 
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highly  subjective  and  difficult  to  conduct  (Nolan,  1974).  The 
difficulties  in  using  this  approach  for  research,  as  argued  by 
Ives  et  al .  (1983),  results  from:  (1)  many  costs  and  benefits 
are  intangible  and  not  easily  recognized  and/or  converted  into 
monetary  equivalents;  (2)  unstructured,  ad  hoc  decision  making 
benefits  are  nearly  impossible  to  objectively  assess;  (3)  even 
when  these  items  are  determined  by  an  organization,  the  data 
are  generally  unrecorded  and  not  available  for  research  ( Ives 
et  al. ,  1983) . 

It  has  been  theorized  that  evaluation  of  the  effectiveness 
or  success  of  an  information  system  is  best  accomplished 
through  the  use  of  decision  analysis.  This  is  based  upon 
studies  which  conclude  that  a  user  will  interact  with  a  system 
in  order  to  obtain  assistance  in  the  decision  making  process 
if  the  system  is  perceived  to  be  useful  in  that  respect.  In 
other  words,  there  will  be  resulting  productivity  benefits. 
(Nolan,  1974)  An  effective  information  system  supports  a  user 
by  retrieving  necessary  information  in  the  proper  format, 
yields  the  required  level  of  detail,  displays  it  at  the  proper 
frequency  and  lends  assistance  to  the  decision-making  process. 
However,  the  decision  analysis  approach  suffers  a  significant 
disadvantage  for  use  in  scientific  research  in  that  it  is  non- 
quantifiable  and  cannot  be  easily  replicated. 
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C.   THE  USER  INFORMATION  SATISFACTION  CONSTRUCT 

The  American  College  Encyclopedia  Dictionary  defines 
satisfaction  as  "the  fulfillment  of  desires,  expectations, 
needs  or  demands".  Organizational  Psychologist  Bernard  Bass 
(1965)  generically  defines  satisfaction  to  mean  "....  the 
extent  to  which  the  item  is  rewarding  to  us  relative  to  how 
much  better  we  might  do  elsewhere  and  what  aspirations  we 
have". 

A  number  of  researchers  have  concluded  that  measuring  User 
Information  Satisfaction  (UIS)  is  a  viable  substitute  for 
measuring  system  success/effectiveness.  Of  note,  however,  are 
two  empirical  studies,  (Aldag  and  Power,  1986;  Gallupe  and 
DeSanctis,  1988).  These  studies  were  conducted  to  find  a 
relationship  between  satisfaction  and  actual  system 
performance  using  objective  third  party  judges.  The  results 
of  these  studies  are  conflicting  and  do  not  support  the 
assumption  that  increased  satisfaction  indicates  actual  system 
performance  increases.  Although  results  have  been  mixed, 
researchers  continue  to  diligently  work  towards  validating  and 
standardizing  UIS  measures.  MIS  practitioners  and  researchers 
agree  that  the  user  is  the  best  gauge  for  determining  the 
success  or  effectiveness  of  an  information  system.  (Neumann 
and  Segev,  1980;  Ives  et  al.,  1983;  Bailey  and  Pearson,  1983; 
Baroudi  and  Orlikowski,  1988). 

The  originators  of  the  concept  of  UIS  were  Cyert  and  March 
(1963).   Their  research  asserts  that  organizational  behavior 
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continually  imposes  upon  the  manager  the  need  for  information. 
If  a  formal  information  system  exits,  the  success  or  failure 
of  that  information  system  to  meet  the  needs  of  the  user 
either  reinforces  or  frustrates  the  user's  sense  of 
satisfaction  with  that  system. 

A  user,  who  presumes  that  required  information  should  be 
readily  available  through  careful  manipulation  of  an 
information  system,  carries  out  the  necessary  steps  to 
retrieve  that  information.  If  the  information  is  indeed 
readily  available,  then  satisfaction  with  the  system  is 
reinforced.  However,  if  the  information  is  not  readily 
available,  the  user  must  modify  the  search  beyond  the 
information  system,  possibly  by  some  other  less  familiar 
procedure.  As  a  result,  the  user  becomes  frustrated. 
Moreover,  an  elaborate  exploration  through  an  information 
system  involves  additional  time  and  effort,  hence,  a  decrease 
in  productivity.  From  the  user's  standpoint,  information 
systems  may  actually  hinder  the  user  in  carrying  out  the  act 
of  decision  making  (Nolan,  1965). 

Successive  use  by  the  user  affords  him/her  the  opportunity 
to  continually  evaluate  the  system.  If  over  a  period  of  time, 
the  user  discovers  that  the  information  system  cannot  retrieve 
specific  information  without  the  frequent  use  of  ad  hoc  means, 
normally  a  response  of  user  dissatisfaction  with  the 
information  system  occurs.  The  project  is  a  failure  if  the 
end  product  does  not  satisfy  those  it  was  designed  to  serve 
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(Powers  and  Dickson,  1973).  If,  however,  the  user  reliably 
obtains  the  required/  requested  information  throuqh  normal 
operation  of  the  information  system,  satisfaction  with  the 
system  will  be  reinforced  and  the  information  system  is 
regarded  as  a  success. 

D.   THE  MEASUREMENT  OF  USER  INFORMATION  SATISFACTION 

User  Satisfaction,  as  a  pervasive  measure  of  system 
effectiveness,  has  gained  wide  acceptance  as  well  as  many 
improvements  along  the  way.  (Ives  and  Olson,  1979;  Igbaria  and 
Nachman,  1990).  Neumann  and  Segev  (1980)  establish  a 
correlation  between  a  user's  reaction  to  factors  having  to  do 
with  satisfaction  and  his/  her  perception  of  their 
organization's  performance.  Swanson  (1974)  empirically  found 
a  high  correlation  between  the  user's  appreciation  of  the 
system  and  his/her  usage  of  its  outputs.  Powers  and  Dickson 
(1973)  concluded  that  user  satisfaction  is  the  most  critical 
factor  for  system  success.  Albeit  there  is  no  standard 
measure  of  satisfaction  in  these  studies,  it  is  maintained 
that  user  information  satisfaction  is  an  indicator  of  system 
usage  and  success. 

The  level  of  user  information  satisfaction  in  the  studies 
previously  referred  to  is  derived  from  a  multitude  of  factors. 
Many  of  the  users  were  asked  to  evaluate  computer  services  in 
terms  of  their  sense  of  satisfaction  (Bailey  and  Pearson, 
1983).   The  factors  measured  in  the  various  studies  included: 
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accuracy,  content,  frequency,  timeliness,  reliability, 
assistance,  adequacy,  accommodation,  communication,  access, 
appreciation  and  flexibility.  Each  of  these  studies  used  a 
measure  that  was  unique  to  that  particular  study  (Ives  et  al . , 
1983),  nevertheless,  taken  as  a  whole,  provide  insight  into  a 
description  of  UIS. 

Validation  of  the  UIS  measures  in  these  studies  is 
limited,  especially  in  handling  threats  to  internal  validity 
(Haga  and  Zviran,  1990).  Bailey  and  Pearson's  (1983)  model 
and  questionnaire  for  measuring  and  analyzing  UIS  seemed  to 
escape  this  deficiency.  It  was  based  on  previous  studies  and 
provides  the  basis  of  an  instrument,  later  refined  (Ives  et 
al . ,  1983),  and  validated  ( Baroudi  and  Orlikowski,  1988)  for 
use.  Although  the  method  employed  to  measure  user 
satisfaction  by  this  tool  is  a  pre-experimental  design  lacking 
pre-test/post-test  or  some  type  of  control  group  (Campbell  and 
Stanley,  1966),  Conrath  and  Mignen  (1990)  assert  that  it 
represents  a  key  contribution  in  the  development  of  a  standard 
instrument  to  measure  user  satisfaction. 

Bailey  and  Pearson  (1983)  described  a  model  proposed  by 
Lawler  and  Wanous  (1972)  for  measuring  user  satisfaction: 

i 


where: 

Rij  =  The  reaction  to  factor  j  by  the  individual  i 
VILi    =  The  importance  of  factor  j   to  individual  i 
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Using  this  model,  the  satisfaction  of  the  user  is  measured 
as  the  weighted  sum  of  the  user's  positive  and  negative 
reactions  against  a  set  of  information  system  factors  (Bailey 
and  Pearson,  1983).  The  user's  perception  of  a  "generically 
good"  information  system  is  described  as  one  composed  of 
factors  which  have  previously  been  deemed  to  be  most  important 
and  a  system  with  which  the  user  is  very  satisfied. 

To  implement  the  model,  a  review  of  22  studies  dealing 

with  computer-user  interface  was  used  to  establish  a  set  of 

factors  representing  the  domain  of  user  satisfaction.  (Bailey 

and  Pearson,  1983).  The  initial  36  factors  generated  from  the 

literature  review  were  expanded  to  39  after  further  review  by 

middle  managers.   They  deduced  that  the  39  factors  decided 

upon  represent  the  domain  of  user  satisfaction  at  a=0.01. 

Bailey  and  Pearson  (1983)  used  four  bipolar  adjective  pairs  on 

a  seven  interval  scale  in  order  for  users  to  explain  and 

measure  their  perception  of  each  factor.   (the  semantic 

differential  technique;  see  Figure  4.1)  The  seven  intervals, 

denoted  by  adverbial  qualifiers,  incrementally  ranged  from 

negative  to  positive  feelings  or  vice  versa.   In  addition  to 

the  four  adjective  pairs,  the  user  was  required  to  test 

internal  consistency  and/or  assign  importance  to  each  factor 

via  two  additional  scales. 

Format  of  Output:  The  material  design  of  the  layout  and 

display  of  the  output  contents. 

good  : : : : : : : :  bad 

simple  : : : : : : : :  complex 
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readable  : : : : : : : :  unreadable 

useful  : : : : : : : :  useless 

To  me,  this  factor  is 

important  : : : : : : : :  unimportant 

Figure  4.1   Illustration  of  questionnaire  form 


Reliability  coefficients  of  the  Bailey  and  Pearson  user 
information  satisfaction  questionnaire  were  determined  to  be 
very  high,  thus  it  was  a  reliable,  valid  instrument  (Bailey 
and  Pearson,  1983).  It  represented  an  important  first  step 
toward  the  development  of  a  valid  and  useful  UIS  measure  ( Ives 
et  al. ,  1983) . 

Deese  ( 1979 )  used  the  Pearson  questionnaire  at  the  Federal 
Computer  Performance  Evaluation  and  Simulation  Center.  He 
professed,  "The  results  identified  problems  that  would  not 
otherwise  have  been  discovered"  and  maintained  that  the  user 
satisfaction  questionnaire  was  a  very  useful  and  worthwhile 
tool  (Deese,  1979). 

Ives,  Olson  and  Baroudi  (1983)  chose  to  undertake  an 
in-depth  assessment  of  the  Pearson  (1983)  questionnaire.  By 
replicating  the  initial  Bailey  and  Pearson  study,  they  too, 
were  able  to  reinforce  the  validity  and  reliability  of  the 
instrument.  They  presented  several  approaches  to  improve  the 
quality  of  the  original  Pearson  instrument. 

The  goal  of  the  suggested  improvements  was  to  establish  a 
standardized  "short  form"  instrument.  Since  Pearson  found 
that  the  importance  scale  provided  no  additional  information, 
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it  was  eliminated  from  the  instrument.  Scales  which  displayed 
undesirable  psychometric  qualities  were  also  eliminated.  The 
number  of  items  scaled  within  a  question  was  reduced  from  four 
to  two  in  order  to  reduce  the  time  to  complete  the 
questionnaire.  All  of  the  scales  in  the  original  Pearson 
instrument  were  scored  positively  to  the  left  and  negatively 
to  the  right  end  of  the  scale.  In  order  to  reduce  undesired 
outcome  produced  when  a  user  simply  marks  down  a  column  of 
responses,  some  of  the  scales  were  reverse  scored,  thereby 
increasing  the  reliability  of  the  measure.  The  resulting 
short  form  was  determined  to  substantially  measure  the  concept 
of  UIS  that  was  originally  proposed  in  the  Pearson  full 
instrument  (Ives  et  al.,  1983). 

The  short  form  measure  (Ives  et  al.,  1983)  was  further 
subjected  to  testing  by  Baroudi  and  Orlikowski  (1988)  to 
examine  its  psychometric  properties.  The  final  short  form 
questionnaire  consisted  of  13  questions  with  two  items 
evaluated  per  question  (see  Appendix  A).  Each  item  is  scored 
on  a  seven  interval  scale,  ranging  from  -3  (dissatisfied)  to 
+3  (satisfied)  with  zero  indicating  a  neutral  response. 

The  total  individual  user  information  satisfaction  score 
is  calculated  by  averaging  the  responses  of  the  2  items  for 
each  question  and  summing  the  scores  of  the  13  questions. 

13  R  +R 

5i_2-  — 9— 
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where: 


St    =  total  individual  user  satisfaction 
F^  =  response  to  the  first  item  scale 
R2  =  response  to  the  second  item  scale 


The  range  of  total  satisfaction  can  be  from  -39  to  +39. 

Three  factors:  electronic  data  processing  ( EDP )  staff  and 
services  (Factor  A),  information  product  (Factor  B),  and 
knowledge  and  involvement  (Factor  C)  were  found  to  comprise 
user  satisfaction  using  factor  analysis  by  Ives,  Olson  and 
Baroudi  (1983).  These  subtotals  are  calculated  as  the  average 
of  the  responses  to  questions  loading  into  a  particular 
factor.  Questions  1,  2,  6,  11,  and  12  load  heavily  into  the 
electronic  data  processing  (EDP)  staff  and  services  factor. 
Questions  7,  8,  9,  10  and  13  load  heavily  into  the  information 
product  factor.  Questions  2,  3,  4,  and  5  load  heavily  into 
the  knowledge  and  involvement  factor.  The  factor  subtotals 
will  range  from  -3  to  +3  in  value.  Averages  are  used  for 
meaningful  comparison  between  the  three  factor  scores. 

Data  gathered  for  the  study  came  from  358  employees, 
mostly  clerical  and  support  personnel,  of  26  New  York  area 
organizations.  Construct  validity  was  determined  first  by 
examining  the  relationship  between  each  scale  and  the  total 
UIS  score.  Factor  analysis  using  varimax  rotation  converged 
to  a  three  factor  solution  accounting  for  68%  of  the  variation 
in  5  iterations.  This  provides  strong  evidence  of  construct 
validity  (Baroudi  and  Orlikowski,  1988). 
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Convergent  validity  was  substantiated  by  comparing  an 
interview  measure  of  satisfaction  with  the  instrument  measure 
in  two  groups.  One  group  of  users  were  generally  pleased  and 
satisfied  with  the  information  system  and  the  other  group  of 
users  were  generally  dissatisfied.  Administration  of  the 
instrument  indicated  a  statistically  significant  difference  by 
t-test  in  the  groups  at  p  <  .001  (Baroudi  and  Orlikowski, 
1988). 

The  total  satisfaction  and  subtotal  scores  reported  a 
reliability  level  above  the  .80  required  for  research.  This 
confirms  the  instrument  to  be  internally  consistent  and 
reasonably  free  from  measurement  error. 

The  work  by  Baroudi  and  Orlikowski  (1988)  and  resulting 
short- form  UIS  instrument  provides  a  reliable  and  valid 
measure  of  user  information  satisfaction.  It  furnishes  a 
meaningful,  standardized  measure  of  overall  satisfaction  with 
an  information  system  as  well  as  specific  information  about 
satisfaction  within  the  sub-factors  of  electronic  data 
processing  (EDP)  staff  and  services,  information  product,  and 
user  knowledge  and  involvement. 
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V.     RESEARCH  METHODOLOGY 


A.  INTRODUCTION 

The  purpose  of  this  research  is  to  identify  the 
characteristics  of  user  information  satisfaction  (UIS)  for 
users  of  the  AQCESS  system  and  compare  the  findings  to  those 
obtained  for  CHCS  (Hurd,  1991).  The  study  is  based  on 
statistical  analysis  of  empirical  data  collected  from  AQCESS 
users  at  the  Naval  Hospital,  Pensacola,  FL  and  Silas  B.  Hays 
Hospital,  Fort  Ord,  CA.  The  following  sections  will  discuss 
the  survey  instrument,  data  collection  methodology  and  methods 
of  statistical  analysis  utilized  in  the  study. 

B.  THE  SURVEY  INSTRUMENT 

Avoiding  the  obstacles  associated  with  developing  a 
comprehensive  survey  questionnaire,  the  previously  developed, 
psychometrically  evaluated  and  validated  short- form 
questionnaire  of  Baroudi  and  Orlikowski  (1988)  was  used 
without  alteration.  A  copy  of  the  complete  survey 
questionnaire  is  included  in  Appendix  A.  The  questionnaire  is 
composed  of  two  sections:  a  demographical  section  and  a  user 
information  satisfaction  section. 
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1 .  Demographic  Section 

The  first  section  of  the  questionnaire  requested 
general  information.  Regarding  questions  1-5,  the  respondents 
were  asked  to  give  their  hospital  division/department,  job 
description,  highest  level  of  education,  age  and  gender, 
respectively.  To  determine  the  user's  experience  level  with 
the  system,  the  sixth  question  asked  for  the  length  of  time, 
in  months,  the  respondent  had  used  AQCESS.  Further 
determination  of  the  user's  experience  was  sought  in  questions 
seven  and  eight.  Question  seven  asked  if  the  user  had 
previously  used  other  computer  systems.  If  the  respondent 
answered  "yes"  to  question  seven,  he/she  was  asked  if  the 
previous  system  was  a  health  care/hospital  information  system. 

2.  User  Information  Satisfaction  Section 

The  second  part  of  the  survey  directly  addressed  the 
issues  of  user  information  satisfaction.  The  instrument  was 
designed  to  obtain  the  user's  opinion  on  how  well  the  AQCESS 
system  functioned.  It  consisted  of  thirteen  questions;  each 
question  having  two  bipolar,  adjective  scales  for  responses. 
Each  item  was  to  marked  with  an  "x"  in  one  of  seven  values, 
ranging  from  -3  (extremely  dissatisfied)  to  +3  (extremely 
satisfied),  with  zero  indicating  a  neutral  response. 

The  questions  asked  can  be  categorized  into  one  of 
three  factors  of  user  satisfaction. 
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Management Information Department/ Information 

Management  Division  (MID/IMP)  Staff:  This  factor  is  the 
respondent's  perception  of  the  attitude  and  responsiveness  of 
the  MID/IMD  staff  as  well  as  their  relationship  with  said 
staff.  The  MID/IMD  staff  provides  only  local  support  and  act 
as  a  liaison  to  the  Defense  Medical  Systems  Support  Center 
( DMSSC )  which  manages  and  administers  the  AQCESS  program. 
Thus,  this  factor  also  takes  into  account  services  provided  by 
DMSSC. 

Information  Product:  This  factor  is  the  respondent's 
perception  of  the  quality  of  output  delivered  by  AQCESS. 

Knowledge  and  Involvement:  This  factor  is  the 
respondent's  perception  of  the  quality  of  training  provided, 
their  understanding  of  the  system,  and  their  participation  in 
its  development/modifications. 

C.   SAMPLE  AND  DATA  COLLECTION 

Naval  Hospital,  Pensacola  and  Silas  B.  Hays  Hospital,  Fort 
Ord  are  two  of  a  multitude  of  U.S.  Navy,  Army,  Air  Force  and 
Coast  Guard  sites  which  operate  AQCESS.  The  sample  population 
then  consisted  of  personnel  from  in/out-patient, 
clinical/technical  and  administration  areas. 

Local  points  of  contact  acted  as  distribution  and 
collection  agents  for  the  surveys.  The  questionnaires,  each 
accompanied  with  a  cover  letter  (Appendix  B),  were  delivered 
to  a  point  of  contact  at  each  site.   The  points  of  contact 
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distributed  500  questionnaires  at  Naval  Hospital,  Pensacola 
and  200  at  Silas  B.  Hays  Hospital,  Fort  Ord  (with  return 
envelopes  and  instructions)  to  all  department  personnel  within 
the  hospital  operating  the  AQCESS  modules.  The  surveys  were 
completed  and  returned  by  the  respondents  to  the  points  of 
contact  in  sealed  envelopes  and  were  then  forwarded  to  the 
researchers  for  analysis. 

There  were  214  questionnaires  returned  from  Pensacola  and 
71  from  Fort  Ord,  of  which  174  and  58  surveys,  respectively, 
had  complete  information.  This  represents  a  final  return  rate 
of  34.8%  and  29%,  respectively. 

D.   STATISTICAL  ANALYSIS  METHODS 

Data  from  completed  surveys  were  coded  and  entered  into  a 
spreadsheet  program.  The  spreadsheet  program  was  used  to 
perform  the  statistical  analysis.  The  specific  procedures 
used  in  the  analysis  will  be  examined  next. 

1.   Respondent's  Satisfaction  Index 

Each   respondent's   total   satisfaction   index   was 
calculated  for  the  13  questions  using  the  following  formula: 

13  R      +R 


where: 

St        =    Satisfaction  Index  for  respondent  i 
Raij  =  Response  to  first  item  scale  of  question  j   for 
respondent  i 
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Rblj  =  Response  to  second  item  scale  of  question  j    for 
respondent  i 

2.  Total  Satisfaction  Index 

The  overall  or  total  satisfaction  index  for  the  survey 
group  is  calculated  by  averaging  the  respondent's  satisfaction 
index  to  find  the  mean. 

3.  Question  Averages 

The  mean  response  to  each  of  the  thirteen  questions  is 
calculated  for  comparison  of  individual  questions. 

4.  Factor  Subtotals 

The  thirteen  questions  can  be  grouped  into  three 
factor  subtotals,  as  defined  by  Ives,  Olson,  and  Baroudi 
(1983):  MID/IMD  staff  and  services  (Factor  A);  Information 
product  ( Factor  B ) ;  and  Knowledge  and  involvement  ( Factor  C ) . 
The  mean  of  the  individual  questions'  averages  is  calculated 
to  find  the  factor  subtotals.  Questions  1,  2,  6,  11,  and  12 
are  used  for  the  MID/IMD  staff  and  services  subtotal. 
Questions  7,  8,  9,  10,  and  13  are  used  for  the  information 
product  subtotal.  Questions  2,  3,  4,  and  5  are  used  for  the 
knowledge  and  involvement  subtotal . 

5.  Comparison  of  Groups 

The  respondents  were  categorized  by  the  type  of  use  of 
the  AQCESS  system.  Three  work  groups  were  identified:  (1) 
Physicians;  (2)  Ancillary;  and  (3)  Administration.  The 
individual  questions'  averages,  total  satisfaction  indexes, 
and  sub- factor  totals  were  compared  for  one  group  verses  the 
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combination  of  the  other  two  groups  using  a  t-test  with  the 
significance  level  of  alpha  =  0.05.  The  t-test  is  used  to 
determine  if  there  is  a  significant  difference  between  the 
arithmetic  mean  value  of  two  groups. 


36 


VI.   DESCRIPTIVE  FINDINGS  (NAVAL  HOSPITAL,  PENSACOLA) 


A.   DEMOGRAPHIC  FINDINGS 

Of  the  174  respondents,  76  were  male  and  98  were  female. 
The  age  of  the  respondents  ranged  from  19  to  77  with  a  mean  of 
36  years  old. 

1.   Work  Center  Distribution 

The  hospital  department  work  centers  reported  were 
in/out-patient  areas  including:  administration  (ie; 
Appointment  and  Scheduling,  Medical  Service  Accounting, 
Emergency  Room  and  Quality  Improvement),  clinics  (ie;  Physical 
Therapy,  OB/GYN,  Pediatrics,  etc.),  laboratory  (ie;  Radiology, 
Pathology,  etc.)  and  other  departments.  Figure  6.1  reflects 
the  distribution  by  work  center. 


PENSACOLA  NAUAL  HOSPITAL 
WORKCENTERS 


OTHER  (2) 


CLINIC  (104) 


AOMINISTDATIUE  (65) 


LABORATORY  (3) 


Figure  6.1 
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2.   Job  Description  Distribution 

The  job  descriptions  reported  are  from  a  multitude  of 
specific  functions  but  can  be  categorized  as: 

1.  Technician  -  a  hospital  corpsman  functioning  in  a 
medical -technical  capacity  (pharmacy  technician, 
laboratory  technician,  clinic  corpsman) 

2.  Physician  -  a  medical  doctor 

3.  Health  Professional  -  a  licensed  medical 
professional  other  than  a  physician  (nurse, 
pharmacist,  physical  therapist,  bio-medical 
officer ) 

4.  Other  -  a  person  not  listed  above  (hospital 
administration  non-medical  clinic  staff) 

Figure   6.2   reflects   the   sample   distribution   by   job 
description. 


PENSACOLA  NAUAL  HOSPITAL 
JOB  DESCRIPTION 


OTHER  (85) 


PHYSICIAN  (22) 


TECHNICIAN  (4?  ) 


HEALTH  DQOrESSIONAL  (20) 


Figure  6.2 
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3.  Work  Group  Distribution 

For  the  purposes  of  this  investigation,  the  study 
population  was  segregated  according  to  the  assumed  primary  use 
of  the  computer  system.  The  categories  provide  a  clear 
separation  of  work  groups  from  an  organizational  and  cultural 
standpoint.  These  groups  are  used  extensively  throughout  the 
study. 

1.  Administrative  -  use  of  the  system  for  indirect 
medical  purposes  (patient  registration, 
appointment  scheduling  and  administrative 
reporting ) 

2.  Ancillary  -  use  of  the  system  for  direct  medical 
support  (prescription  filling,  laboratory  specimen 
processing ) 

3.  Physician  -  use  of  the  system  by  physicians 
(accessing  patient  historical  data,  performance 
reports ) 

Figure   6.3   represents   the   distribution   of   respondents 
according  to  their  main  use  of  the  system. 

4.  Education  Distribution 

The  level  of  education  reported  by  the  respondents 
represents  a  well  educated  user  population.   A  little  more 
than  79  percent  of  the  users  have  had  at  least  some  college 
education  with  nearly  31  percent  having  completed  a  Bachelor, 
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Master,     Medical    or    PhD    degree.        Figure    6.4    represents    the 
breakdown  by  education. 


PENSACOLA    NAUAL    HOSPITAL 
WORK  GROUP 


ANCILLARY  C67 > ■ 


PHYCISIAN  (22) 


AOMINISTRATIUE  C85 > 


Figure  6.3 


PENSACOLA    NAUAL    HOSPITAL 

EDUCATION 

r-OOCTORAL    OEGQEE    Cl> 

HIGH   SCHOOL    l3Qi-y^"^ 

1— SOME   COLLE6E    <84 

MEDICAL    OEGDEE     (22  )  -~W§"ffflP 

HASTERS  DEGREE    (  13  )  ^^?|||ss3    ^/ 

BACHELORS  DEGREE    <16>-' 

Figure  6.4 
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B.  COMPUTER  SYSTEM  USE 

The  length  of  time  the  respondents  reported  as  having  used 
the  AQCESS  computer  system  ranged  from  one  to  72  months  with 
a  mean  of  21  months.  Of  the  174  respondents,  63  had  used  a 
computer  system  previously.  58  of  the  63  users  had  used  a 
health  care  information  system  before  using  AQCESS.  Although 
many  of  the  respondents  had  experience  with  computers,  this 
was  the  first  exposure  to  a  health  care  information  system  for 
67  percent  of  the  users. 

C.  SATISFACTION  FINDINGS 

The  survey  results  for  user  satisfaction  are  compared  for 
the  three  work  groups  in  three  areas:  1)  overall  satisfaction, 
2)  responses  to  individual  questions,  3)  three  group  factors: 
MID  staff  and  services;  information  product;  knowledge  and 
involvement.  The  actual  values  of  responses  to  individual 
questions  reported  in  the  survey  may  be  significant  in 
themselves.  However,  the  relative  scores  are  useful  for 
comparison  among  various  work  groups,  organizations  and 
information  systems.  ( Baroudi  and  Orlikowski,  1988)  The 
administrative  group  composes  nearly  48  percent  of  the  survey 
population,  the  ancillary  group  composes  40  percent  and  the 
remainder  consists  of  physicians. 
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1.   Overall  Satisfaction  (Pensacola) 

The  overall  satisfaction  was  determined  by  the  mean  of 
the  sum  of  the  174  survey  responses  to  the  thirteen  survey 
questions  from  the  work  groups  at  Pensacola.  The  mean  overall 
satisfaction  index  of  the  combined  work  groups  was  11.16  on  a 
scale  ranging  from  -39  to  +39.  Figure  6.5  illustrates  a 
comparison  of  the  overall  satisfaction  index  of  the  work 
groups  at  Pensacola.  The  survey  shows  that  the  administrative 
group  displays  the  highest  overall  satisfaction  (13.81), 
closely  followed  by  ancillary  (13.73),  with  physicians 
registering  the  least  overall  satisfaction  (-4.16).  Using  a 
t-test,  the  level  of  satisfaction  for  physicians  was 
statistically  significant  lower  at  alpha  =0.05  than  remaining 
groups . 


OUERALL  SATISFACTION 
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Figure   6.5 
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2.   Individual  Questions 

The  13  questions  on  the  survey  requires  two  responses 
from  the  respondent.  A  number  of  the  response  scales  are 
reverse  scored  to  nullify  the  effects  of  a  respondent  who 
simply  marks  down  one  column  of  the  questionnaire  ( Baroudi  and 
Orlikowski,  1988).  Fiqure  6.6  displays  the  averaqe  level  of 
satisfaction  for  each  question,  by  work  group,  for  Pensacola. 


Individual  Question  Response 
By  Work  Group 
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Figure  6.6 

It  is  easily  discernable  that  some  questions  exhibit  a 
relatively  high  or  low  level  of  satisfaction  when  compared  to 
the  other  questions.  Questions  1,  6  and  11  indicate  a  high 
level  of  user  satisfaction.  These  questions  deal  with  issues 
concerning  the  Management  Information  Department  (MID)  staff. 
However,  questions  2,  3  and  12  stand  out  as  exhibiting  a 
relatively  low  satisfaction  index.   Questions  2  and  12  are 
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concerned  with  software  development/modification  and  question 
3,  training.  The  following  discussion  will  examine  each 
question  by  work  group. 

Figure  6.7  illustrates  Pensacola  physician's  responses  to 
individual  questions  as  they  compare  to  the  remainder  of  the 
survey  group.  Figure  6.8  represents  the  ancillary  group's 
responses  as  compared  to  the  remainder  of  the  survey  groups . 
Figure  6.9  is  the  administrative  group's  responses  as  compared 
to  the  remainder  of  the  survey  groups. 


Individual  Question  Response 
PHYSICANS  vs  OTHERS 

2 


x  1.5 

I  . 

J  0.5 

1  ° 


2  -0.5 

a 

/i       -! 

-1.5 


1    2    3    4    5    6    ?    B    9    10  11  12  13 
Question    Number 


|  Phystelon  ^  OTHERS 


Figure  6.7 

a.   Question  1  -  Relationship  with  the  Management 
Information  Department   (MID)   Staff 

This  question  measures  the  relationship  between 
users  and  the  Management  Information  Department.  The  level  of 
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satisfaction  towards  the  MID  staff  is  relatively  high.  It  has 
the  highest  satisfaction  index.  The  administrative  group 
showed  the  highest  level  of  satisfaction  of  the  three  groups. 
The  ancillary  group  was  slightly  lower.  The  physicians 
revealed  a  statistically  significant  lower  difference  at  alpha 
=  0.05. 
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b.  Question  2  -  Processing  of  Requests  for  Changes 
to  Existing  Systems 

This  question  is  concerned  with  the  length  of  time 
required  and/or  timeliness  of  changes  to  AQCESS.  This  was  the 
lowest  scoring  question  which  indicates  a  user  perceived 
problem.  Physicians  were  the  least  satisfied  with  a 
statistically  significant  lower  difference  at  alpha  =  0.05. 
The  administrative  users  displayed  a  statistically  significant 
higher  difference  at  alpha  =  0.05. 

c.  Question  3   -  Degree  of  Training  Provided  to  Users 
The  response  to  this  question  indicates  the  user's 

perceived  level  of  training  to  use  the  system.  The  level  of 
satisfaction  of  the  administrative  and  ancillary  groups  was 
nearly  equal.  The  physicians,  however,  showed  a  statistically 
significant  lower  difference  at  alpha  =  0.05. 

d.  Question  4  -  User's  Understanding  of  the  System 
The  response  to  this  question  is  the  respondent's 

self-reported  assessment  of  his/her  ability  to  interact  with 
the  system.  Physicians  were  statistically  significant  lower 
at  alpha  =  0.05.  Whereas,  the  administrative  group  showed  a 
statistically  significant  higher  difference  at  alpha  =  0.05. 
The  ancillary  group  indicated  they  were  substantially  less 
satisfied  with  the  amount  of  training  they  received  (question 
2)  as  compared  to  their  understanding  of  the  system. 
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e.  Question  5   -   User's  Feeling  of  Participation 

The  responses  to  this  question  yield  a  measurement 
of  the  user's  perception  of  participation  in  the  system.  The 
administrative  group  had  a  statistically  significant  higher 
perception  of  participation  than  the  other  groups. 
Conversely,  physicians  had  a  statistically  significant  lower 
difference  at  alpha  =  0.05.  The  ancillary  group's  feeling  of 
participation  was  slightly  lower  than  the  administrative,  but 
not  statistically  significant. 

f.  Question   6  -  Attitude  of  the  Management 
Information  Department    (MID)   Staff 

The  intent  of  this  question  is  to  gauge  the  user's 
perception  of  the  attitude  of  the  MID  staff.  In  particular, 
the  staff's  attitude  towards  rendering  services.  The  response 
to  this  question  was  positive  for  all  three  groups.  The 
administrative  group  was  statistically  significant  higher 
while  the  physicians  were  statistically  significant  lower  at 
alpha  =  0.05.  However,  this  question  ranked  third  among  the 
physician's  positive  responses. 

g.  Question  7  -  Reliability  of  Output  Information 
The  user's  response  to  this  question  indicates 

his/her  view  of  consistency  and  dependability  of  the  output 
information.  Administrative  and  ancillary  groups  responses 
were  positive  and  approximately  the  same.  Physicians' 
response  was  statistically  significant  lower  at  alpha  =  0.05. 
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h.    Question  8  -  Relevancy  of  Output  Information    (to 
Intended  function) 

This  question  measures  the  degree  of  compatibility 
between  what  the  user  wants  or  requires  and  what  is  provided 
by  the  information  products  and  services.  Administrative 
group  showed  a  statistically  significant  higher  difference  at 
alpha  =  0.05.  The  physicians,  however,  displayed  a 
statistically  significant  lower  difference  at  alpha  =  0.05. 
The  ancillary  group  was  slightly  lower  than  the  administrative 
group  with  no  statistical  significance. 

1.    Question  9  -  Accuracy  of  Output  Information 

The  user's  opinion  of  correctness  of  the  output 
information  is  captured  by  this  question.  Administrative 
group  was  statistically  significant  higher  while  physicians 
were  statistically  significant  lower.  Ancillary  was  not 
statistically  significant,  but  was  equally  satisfied  as  the 
administrative  group. 

j.    Question  10  -  Precision  of  Output  Information 

Precision  of  output  information  is  the  variability 
of  output  information  from  that  which  it  purports  to  measure. 
The  administrative  and  ancillary  groups  were  not  statistically 
significant,  but  the  administrative  group's  response  was 
approximately  twice  that  of  the  ancillary  group's.  The 
physicians  had  a  statistically  significant  lower  difference  at 
alpha  =  0.05. 
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Jc.  Question    11    -    Communication    with    the    Management 
Information  Department    (MID)    Staff 

The  response  to  this  question  is  a  basis  for 
determining  user  perception  of  the  manner  and  methods  of 
information  exchange  between  users  and  the  MID  staff.  The 
results  for  the  administrative  and  ancillary  groups  were  not 
statistically  significant,  but  instead,  the  scores  were  nearly 
even.  Physicians  did,  however,  show  a  statistically 
significant  lower  difference  at  alpha  =  0.05. 

1.    Question     12     -     Time     Required     for     New     Systems 

Development 

This  question  is  designed  to  assess  the  user's 
perception  of  the  elapsed  time  between  his/her  requests  for 
new  applications  and  the  design,  development  and  the 
implementation  of  the  application  systems  by  the  contractor  or 
outside  service  representative.  The  administrative  and 
ancillary  groups  were  nearly  neutral.  The  physicians,  however 
displayed  a  statistically  significant  lower  difference  at 
alpha  =  0.05. 

m.  Question  13   -  Completeness  of  Output 

The  response  to  this  question  was  intended  to  help 
measure  the  user's  impression  of  the  comprehensiveness  of  the 
content  of  the  output  information.  The  administrative  group 
were  statistically  significant  difference  higher,  while  the 
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physicians  were  statistically  significantly  lower  at  alpha  = 
0.05. 

3.   Grouped  Factors 

The  original  user's  information  satisfaction  survey 
instrument  sub-divided  the  thirteen  questions  into  three 
grouped  factors.  These  factors  are  designated  as  EDP 
(MID/IMD)  staff  and  services,  information  system  product,  and 
knowledge  and  involvement.  (Ives,  Olson  and  Baroudi, 1988 )  The 
three  factors  are  calculated  from  the  averages  of  the 
component  questions.  MID  staff  and  services  (Factor  A)  is 
sub-divided  into  two  separate  components:  local  MID  staff 
functions  (MID)  and  contractor  or  Defense  Medical  Systems 
Support  Center  (DMSSC)  functions  (Con).  Factor  A  (MID)  is 
derived  from  questions  1,  6, and  11;  Factor  A  (Con)  from 
questions  2  and  12;  information  system  product  (Factor  B)  from 
questions  7,  8,  9,  10  and  13;  and  knowledge  and  involvement 
(Factor  C)  from  questions  2,  3,  4,  and  5  (Hurd,1991). 

The  administrative  group's  and  physicians'  level  of 
satisfaction  were  statistically  different  at  significance 
level  alpha  =  0.05.  The  administrative  group  was  the  most 
satisfied  and  the  physicians  were  the  least  satisfied  of  all 
the  groups.  Figure  6.10  represents  the  work  groups'  level  of 
satisfaction  with  relation  to  sub-total  factors.  Figure  6.11 
represents  factor  satisfaction  of  physicians  compared  to  the 
other  work  groups. 
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Figure   6.12   represents   factor   satisfaction   of 
ancillary  personnel  compared  to  the  other  work  groups. 
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Figure    6.13    represents    factor    satisfaction    of 
administrative  personnel  compared  to  the  other  work  groups. 
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Figure  6.14  represents  the  group  factor  satisfaction 
indices  of  administrative  personnel  compared  to  ancillary 
personnel . 
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Figure  6.14 

a.  Factor  A   (Local  MID  Staff  and  Services) 

This  factor  is  the  respondent's  self -reported 
assessment  of  the  attitude  and  responsiveness  of  the  local  MID 
staff  and  the  services  they  provide,  as  well  as,  the  quality 
of  the  relationship  with  the  MID  personnel  ( Baroudi  and 
Orlikowski ) .  As  seen  in  Figure  6.10,  the  administrative 
personnel  displayed  the  highest  level  of  satisfaction  with  a 
statistically  significant  difference  at  alpha  =  0.05.  The 
lowest  level  of  satisfaction  was  displayed  by  the  physicians, 
however,  it  is  exceptionally  high  in  comparison  with  their 
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responses   to   the   other   factors.     The   physicians   were 
statistically  significant  at  alpha  =  0.05. 

b.  Factor  A   (Contractor,    DMSSC) 

This  factor  represents  the  responsiveness  and 
services  of  outside  software  programmers  and  management.  It 
received  the  lowest  level  of  satisfaction  of  any  factor.  All 
work  groups  rated  this  factor  extremely  low,  with 
administrative  personnel  recording  the  highest  level  of 
satisfaction  and  the  physicians  scoring  the  lowest  level  of 
satisfaction.  The  administrative  group  showed  a  statistically 
significant  higher  difference  while  the  physicians  showed  a 
statistically  significant  lower  difference  at  alpha  =  0.05. 

c.  Factor  B  -  Information  Product 

This  factor  represents  the  users  perception  of  the 
quality  of  output  delivered  by  the  information  system  (Baroudi 
and  Orlikowski,  1988).  The  administrative  group  scored  the 
highest  level  of  satisfaction  with  a  statistically  significant 
difference  at  alpha  =  0.05.  The  physicians  recorded  the 
lowest  level  of  satisfaction  with  a  statistically  significant 
difference  at  alpha  =  0.05. 

d.  Factor  C  -  Knowledge  and  Involvement 

This  factor  represents  the  respondents'  self- 
reported  assessment  of  the  quality  of  training  provided,  their 
understanding  of  the  system,  and  their  participation  in  its 
development  (Baroudi  and  Orlikowski,  1988).    There  was  a 
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statistically  significant  difference  between  the  level  of 
satisfaction  of  the  administrative  group  and  the  physicians  at 
alpha  =  0.05.  The  administrative  group  and  ancillary  group 
rated  the  satisfaction  level  nearly  the  same.  The  physicians 
scored  this  factor  the  lowest  of  all  the  group  factors. 
4.   Time  of  System  Use 

Igbaria  and  Nachman  (1990)  found  that  users  with 
better  computer  background  are  likely  to  be  more  confident  in 
their  ability  to  use  the  information  system  and  more  satisfied 
with  experience.  Generally,  users  become  more  skilled  with  an 
information  system  over  time.  Therefore,  the  assumption  is 
that  the  respondents  who  use  the  system  the  longest  will  be 
more  experienced  and  record  a  greater  level  of  satisfaction. 

The  respondents  were  separated  into  subgroups  of  users 
with  less  than  six  months,  six  months  to  11  months,  and 
greater  than  11  months  experience.  Almost  sixty  percent  of 
the  respondents  had  11  months  or  greater  experience  with  the 
AQCESS  system.  Figure  6.15  represents  the  breakdown  of 
respondents  by  time  of  system  use. 

The  respondents  who  used  the  system  more  than  or  equal 
to  11  months  generated  the  lowest  level  of  user  satisfaction. 
Conversely,  the  respondents  with  6  to  11  months  of  system  use 
recorded  the  highest  level  of  satisfaction  with  the  system. 
Regression  analysis  indicated  no  correlation  (r  <  0.02) 


55 


between  the  time  of  system  use  and  level  of  user  satisfaction 
with  the  system  for  any  factor. 
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Figure  6.16  shows  the  level  of  satisfaction  with  AQCESS 
verses  time  of  system  use. 
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Breaking  the  work  groups  down  by  time  of  system  use, 
there  was  still  no  correlation  between  time  of  system  use  and 
level  of  satisfaction  of  any  factor  by  regression  analysis. 
The  following  discussion  will  describe  the  level  of  user 
satisfaction  for  the  work  groups  over  time. 

The  physicians  with  less  than  six  months  system  use 
had  a  satisfaction  index  of  2.625.  Unlike  the  ancillary  and 
administrative  groups,  the  physicians'  level  of  satisfaction 
decreased  to  -3.25  between  six  to  11  months  experience.  The 
level  of  satisfaction  for  the  physicians  with  more  than  11 
months  experience  show  a  decrease  at  -6.36,  a  downward  trend 
consistent  with  the  other  work  groups.  Figure  6.17 
illustrates  the  physician's  level  of  satisfaction  with  respect 
to  time  of  system  use. 
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The  ancillary  group's  level  of  satisfaction  displays 
a  similar  trend  to  the  administrative  group.  The  satisfaction 
index  for  users  with  less  than  six  months  experience  was 
slightly  lower  (18.17)  than  users  with  six  to  11  months 
experience  whose  level  of  satisfaction  rose  to  19.8. 
Surprisingly,  the  level  of  satisfaction  for  users  with  more 
than  11  months  of  experience  drops  dramatically  to  9.4. 
Figure  6.18  illustrates  the  ancillary  group's  level  of 
satisfaction  with  respect  to  time  of  system  use. 
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Figure  6.18 

Although  the  trends  for  both  the  administrative  and 
the  ancillary  group's  level  of  satisfaction  are  nearly  the 
same,  the  administrative  group's  level  of  satisfaction  amongst 
the  user  subgroups  is  more  pronounced.  The  satisfaction  index 
for  users  with  less  than  six  months  experience  was  lower  at 
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15.6  than  users  with  six  to  11  months  experience  whose  level 
of  satisfaction  rises  steeply  to  22.35.  The  level  of 
satisfaction  for  users  with  more  than  11  months  of  experience 
drops  sharply  to  9.4.  Figure  6.19  illustrates  the 
administrative  group's  level  of  satisfaction  with  respect  to 
time  of  system  use. 
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VII.   DESCRIPTIVE  FINDINGS  (SILAS  B.  HAYS  HOSPITAL,  FORT  ORD) 


A.   DEMOGRAPHIC  FINDINGS 

Of  the  57  respondents,  10  were  male  and  47  were  female. 
The  age  of  the  respondents  ranged  from  19  to  65  with  a  mean  of 
34  years  old. 

1 .   Work  Center  Distribution 

The  hospital  department  work  centers  reported  were 
in/out-patient  areas  including:  other  (ie;  Patient 
Administration  Division,  Quality  Assurance,  etc. )  and  clinics 
(ie;  OB/GYN,  Pediatrics,  etc.).  Figure  7.1  reflects  the 
distribution  by  work  center. 
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Figure   7.1 
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2.   Job  Description  Distribution 

The  job  descriptions  reported  are  from  a  multitude  of 
specific  functions  but  can  be  categorized  as: 

1.  Technician  -  a  hospital  specialist  functioning  in 
a  medical-technical  capacity  (laboratory 
technician,  clinical  technician,  medic) 

2.  Other  -  a  person  not  listed  above  (hospital 
administration  non-medical  clinic  staff) 

Figure  7.2  reflects  the  distribution  by  job  description. 
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Figure   7.2 

3.   Work  Group  Distribution 

For  the  purposes  of  this  investigation,  the  study 
population  was  segregated  according  to  the  assumed  primary  use 
of  the  computer  system.   The  categories  provide  a  clear 
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separation  of  work  groups  from  an  organizational  standpoint. 
These  groups  are  used  extensively  throughout  the  study  of  the 
Silas  B.  Hays  Hospital  sample  (Fort  Ord ) . 

1.  Administrative  -  use  of  the  system  for  indirect 
medical  purposes  (patient  registration, 
appointment  scheduling  and  administrative 
reporting ) 

2.  Ancillary  -  use  of  the  system  for  direct  medical 
support  (prescription  filling,  laboratory  specimen 
processing) 

Figure  7.3  represents  the  population  of  users/respondents 
according  to  their  main  use  of  the  system. 
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62 


4.   Education  Distribution 

The  level  of  education  reported  by  the  users 
represents  a  fairly  well  educated  user  population. 
Approximately  67  percent  of  the  users  have  at  least  had  some 
college  education.  12  percent  completed  a  Bachelors  or 
Masters  degree  with  5  percent  having  worked  on  a  Masters 
degree.   Figure  7.4  represents  the  breakdown  by  education. 
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Figure  7 . 4 

B.   COMPUTER  SYSTEM  USE 

The  length  of  time  users  reported  as  having  used  the 
AQCESS  computer  system  ranged  from  one  month  to  72  months  with 
a  mean  of  19.4  months.  Of  the  57  respondents,  22  had  used  a 
computer  system  previously.  16  of  the  57  users  had  used  a 
health  care  information  system  before  using  AQCESS.   Although 
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many  of  the  respondents  had  experience  with  computers,  this 
was  the  first  exposure  to  a  health  care  information  system  for 
72  percent  of  the  users. 

C.   SATISFACTION  FINDINGS 

The  survey  results  for  user  satisfaction  are  compared  for 
the  two  work  groups  in  three  areas:  1)  overall  satisfaction, 
2)  responses  to  individual  questions,  3)  three  group  factors: 
Information  Management  Division  (IMD)  staff  and  services; 
information  system  product;  knowledge  and  involvement.  The 
actual  values  of  responses  to  individual  questions  reported  in 
the  survey  may  be  significant  in  themselves.  However,  the 
relative  scores  are  useful  for  comparison  among  various  work 
groups,  organizations  and  information  systems.  (Baroudi  and 
Orlikowski,  1988)  The  administrative  group  composes  nearly  97 
percent  of  the  survey  population  and  the  remainder  consists  of 
the  ancillary  group. 

1.   Overall  Satisfaction  (Fort  Ord) 

Overall  satisfaction  is  measured  by  the  mean  of  the 
sum  of  the  responses  to  the  survey  questions.  The  mean  survey 
score  index  was  10.47  on  a  scale  ranging  from  -39  to  +39. 
Figure  7.5  presents  a  comparison  of  the  overall  satisfaction 
index  of  both  groups  and  the  overall  mean  satisfaction.  The 
survey  shows  that  the  ancillary  group  displays  the  highest 
overall  satisfaction  index  (13.0),  with  the  administrative 
group  recording  the  lower  overall  satisfaction  (10.38).  Using 


64 


a  t-test,  there  was  no  significant  difference  between  the 
ancillary  group  and  the  administrative  group  at  alpha  =  0.05. 
2.   Individual  Questions 

The  13  questions  on  the  survey  require  two  responses 
each  from  the  respondent.  A  number  of  the  response  scales  are 
reverse  scored  to  nullify  the  effects  of  a  respondent  who 
simply  marks  down  one  column  of  the  questionnaire  ( Baroudi  and 
Orlikowski,  1988).  Figure  7.6  shows  the  average  level  of 
satisfaction  for  individual  questions  by  work  group. 

Similar  to  the  results  discussed  in  chapter  six, 
questions  1,  6  and  11  indicate  a  high  level  of  user 
satisfaction.  These  questions  deal  with  issues  concerning  the 
IMD  staff.  However,  questions  2,  3  and  12  stand  out  as 
exhibiting  a  relatively  low  satisfaction  index.  Questions  2 
and  12  are  concerned  with  DMSSC  services  and  modifications  and 
question  3,  training.  The  following  discussion  will  examine 
each  question  by  work  group. 

a.  Question  1   -  Relationship  with  the  Information 
Management  Division   (IMD)   Staff 
As  noted  earlier,  the  level  of  satisfaction  towards 
the  IMD  staff  is  relatively  high.   Although  not  statistically 
significant  at  alpha  =  0.05,  the  ancillary  group  showed  the 
higher  level  of  satisfaction  between  the  two  groups. 
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OUERALL  SATISFACTION 
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b.  Question  2   -  Processing  of  Requests  for  Changes 
to  Existing  Systems 

This  question  received  the  lowest  satisfaction 
index  rating.  The  response  was  neutral  for  the  ancillary 
group  and  negative  for  the  administrative  group.  A  t-test 
indicated  no  statistically  significant  differences  at  alpha  = 
0.05. 

c.  Question  3   -  Degree  of  Training  Provided  to  Users 
The  level  of  satisfaction  for  the  ancillary  group 

was  higher  than  that  of  the  administrative  group  but  not 
statistically  significant  difference  higher  at  alpha  =  0.05. 

d.  Question  4  -  User's  Understanding  of  the  System 
There  was  no  statistically  significant  difference 

between  the  administrative  and  ancillary  groups  at  alpha  = 
0.05.  Again,  the  ancillary  group  displayed  a  higher 
satisfaction  index. 

e.  Question  5  -  User's  Peeling  of  Participation 

In  response  to  this  question,  the  administrative 
and  the  ancillary  groups  responded  positively  with  no 
statistically  significant  difference  at  alpha  =  0.05. 

f .  Question   6  -  Attitude  of  the  Information 

Management  Division   (IMD)   Staff 
The  response  to  this  question  (the  second  highest 
rated   question)   was   favorable   with   no   statistically 
significant  difference  at  alpha  =  0.05. 
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g.    Question  7   -  Reliability  of  Output  Information 

The  responses  were  similar  with  no  statistically 
significant  differences  between  the  groups. 

h.    Question   8    -   Relevancy   of  Output    Information    (to 

intended  function) 

Both  groups  responded  comparably  without 
statistically  significant  differences. 

i.    Question  9  -  Accuracy  of  Output  Information 

The  ancillary  group  was  neutral  regarding  this 
item.  The  administrative  group  was  slightly  more  positive, 
but  there  were  no  statistically  significant  differences  at 
alpha  =  0.05. 

j.  Question  10  -  Precision  of  Output  Information 

The  response  by  both  groups  was  somewhat  favorable 
with  the  administrative  group's  satisfaction  index  being  a 
little  higher.  There  were,  however,  no  statistically 
significant  differences. 

k.    Question  11   -  Communication  with  the  Information 

Management  Division   (IMD)   Staff 

All  questions  concerning  the  IMD  staff  received 
high  satisfaction  levels  from  both  groups  (highest  rated 
question  response).  Neither  of  the  groups'  responses  were 
statistically  different  by  t-test  at  alpha  =  0.05. 
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I .  Question  12      -  Time     Required     for     New     Systems 
De ve 1 opmen t 

The  groups  were  nearly  neutral  regarding  this 
issue.  There  were  no  statistically  significant  differences  at 
alpha  =  0.05. 

m.    Question   13   -   Completeness   of  Output 

Both    groups    responded    similarly    with    no 
statistically  significant  differences  at  alpha  =  0.05. 
3 .   Grouped  Factors 

As  discussed  in  chapter  VI,  the  questions  are  grouped 
into  three  factors  that  comprise  user  satisfaction.  These 
factors:  IMD  staff  and  services,  information  system  product, 
and  knowledge  and  involvement,  are  calculated  through  averages 
of  the  component  questions.  IMD  staff  and  services  (Factor  A) 
is  sub-divided  into  two  separate  components:  local  IMD  staff 
functions  (IMD)  and  contractor /DMSSC  functions  (CON) . 

Neither  of  the  work  groups'  satisfaction  level  for  any 
of  the  factors  were  statistically  different  at  significance 
level  alpha  =  0.05.  Figure  7.7  represents  the  ancillary  and 
administrative  groups'  level  of  satisfaction  in  relation  to 
the  group  factors. 

a.  Factor  A    (Local   MID   Staff  and  Services) 

In  Figure  7.7,  the  ancillary  group  displayed  the 
highest  level  of  satisfaction  but  it  was  not  statistical!/ 
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significant   different    from   the   administrative   group   at   alpha 
=    0.05. 
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Figure  7.7 

b.  Factor  A   (Contractor  Services) 

This  factor  received  the  highest  level  of 
satisfaction  of  any  factor.  The  ancillary  group  was  not 
statistically  significant  different  from  the  administrative 
group  at  alpha  =  0.05. 

c.  Factor  B  -  Information  Product 

This  factor  is  the  second  highest  rated  of  the  four 
and  there  is  no  statistically  significant  difference  between 
the  ancillary  and  administrative  groups  at  alpha  =  0.05. 
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d.    Factor   C  -  Knowledge   and  Involvement 

This  factor  showed  no  statistically  significant 
difference  between  the  level  of  satisfaction  for  the  ancillary 
or  administrative  groups. 
4 .   Time  of  System  Use 

As  stated  in  chapter  VI,  it  is  expected  that  the  more 
experienced  users  will  exhibit  a  greater  level  of 
satisfaction. 

Figure  7.8  displays  the  separation  of  users  into 
subgroups  having  less  than  six  months  experience,  six  to  11 
months,  and  greater  than  11  months. 
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The  ancillary  group  had  no  users  with  less  than  six 
months  nor  greater  than  11  months  of  system  use.   Therefore 
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there  is  really  no  trend  to  report  for  this  group.  However, 
it  is  worthy  to  note  that  the  two  respondents  displayed  a  high 
level  of  satisfaction  with  an  index  of  13.0.  Figure  7.9 
depicts  the  ancillary  users'  satisfaction  index  versus  the 
time  of  system  use. 
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Figure  7.9 

The  administrative  group's  level  of  satisfaction  tends 
to  increase  with  experience  and  time  of  use.  Respondents, 
with  less  than  six  months  use,  recorded  a  satisfaction  index 
of  5.3.  As  the  users  gained  experience  (six  to  11  months), 
the  level  of  satisfaction  increased  to  10.8.  For  users  with 
greater  than  11  months  experience,  the  satisfaction  index 
elevated  to  12.06.  Figure  7.10  represents  the  administrative 
users'  satisfaction  index  versus  the  time  of  system  use. 
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VIII.   COMPARISON  OF  RESULTS  FROM  FORT  ORD  AND  PENSACOLA 


A.   INTRODUCTION 

When  the  user  satisfaction  survey  questionnaire  is 
assessed  within  a  single  organization,  the  UIS  scores  are 
indicative  of  the  general  level  of  user  satisfaction  with  a 
specific  information  system.  The  scores,  positive  or 
negative,  are  on  its  own  an  important  finding.  However,  the 
scores  are  useful  for  comparisons  across  different  users  of 
the  same  information  system  (to  pinpoint  the  problems 
particular  users  may  be  experiencing) ,  as  well  as  for 
comparisons  across  various  information  systems  (to  highlight 
specific  information  systems  that  may  be  problematic) . 
(Baroudi  and  Orlikowski,  1988) .  Silas  B.  Hays  Army  Hospital 
(Fort  Ord)  and  Pensacola  Naval  Hospital  are  two  different 
organizations  with  similar  information  systems  (AQCESS) .  This 
chapter  will  compare  the  user  information  satisfaction  (UIS) 
measurements  recorded  from  Fort  Ord  with  the  UIS  measurements 
recorded  from  Pensacola.  In  a  later  chapter,  these  findings 
will  be  analyzed  to  pinpoint  any  agreements  or  disagreements 
amongst  the  work  groups  of  these  organizations. 
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B.   OVERALL  USER  SATISFACTION 

The  impressions  of  the  respondents  from  Pensacola  and 
Fort  Ord  towards  their  information  system  are  comparable. 
Using  a  t-test,  the  satisfaction  level  of  the  two 
organizations,  has  no  significant  difference  at  alpha  =  0.05. 
The  Fort  Ord  respondents  did  register  a  slightly  higher  level 
of  satisfaction  at  10.47  which  is  inconsequential  compared  to 
Pensacola' s  overall  satisfaction  index  at  9.02.  Together  their 
overall  level  of  satisfaction  is  approximately  9.64.  Figure 
8.1  describes  the  extent  of  the  difference  in  the  satisfaction 
level  of  the  two  organizations. 
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The  combined  work  groups'  overall  satisfaction  was 
determined  by  combining  the  mean  of  the  sum  of  their 
corresponding  satisfaction  indices  measured  from  the  survey 
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questions  of  the  57  respondents  from  Fort  Ord  and  the  174 
respondents  from  Pensacola.  The  survey  results,  for  the 
combined  administrative  group,  display  the  highest  overall 
satisfaction  index  at  11.87,  followed  by  the  combined 
ancillary  group's  level  of  satisfaction  at  9.46.  Fort  Ord  did 
not  have  any  survey  responses  from  physicians.  Thus  the 
overall  satisfaction  level,  -4.16,  for  the  physicians'  is 
obtained  solely  from  the  Pensacola  physicians.  As  noted  in 
chapter  six,  the  physician's  level  of  satisfaction  has  a 
statistically  significant  lower  difference  than  the  other 
groups  at  alpha  =  0.05.  Figure  8.2  describes  the  overall 
level  of  satisfaction  of  the  work  groups  from  Fort  Ord  and 
Pensacola. 
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1.   Fort   Ord's   and   Pensacola's   Individual   Question 
Responses 

The  method  used  to  calculate  the  values  of  the 
individual  question  responses  was  described  in  earlier 
chapters.  Together,  Fort  Ord  and  Pensacola  display  a  similar 
pattern  of  satisfaction  levels  for  individual  question 
responses  as  they  did  separately.  Still,  questions  1,  6  and 
11  (deals  with  issues  concerning  MID/IMD)  show  the  highest 
level  of  satisfaction.  Consistent  with  previous  reports, 
questions  2,  3  and  12  (deals  with  DMSSC  services  and 
management)  has  the  lowest  rated  indices.  Figure  8.3 
illustrates  the  combined  survey  question  responses  for  Fort 
Ord  and  Pensacola. 
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The  combined  work  groups  display  a  similar  pattern  of 
satisfaction  levels  for  individual  question  responses  as 
previously  noted.  Questions  1,  6  and  11  consistently  rate  as 
having  the  highest  level  of  satisfaction,  while  questions  2, 
3  and  12  consistently  rate  as  having  the  lowest  level  of 
satisfaction  by  the  respondents  of  the  combined  work  groups. 
Each  question  will  be  discussed  further  in  the  following 
sections.  Since  the  physician's  responses  are  solely  from 
Pensacola  and  are  not  changed  by  combining  the  work  group 
responses  from  Fort  Ord,  they  will  not  be  discussed  as  part  of 
the  work  group.  They  will  be  reflected  in  the  overall 
comparison  of  the  individual  question  responses  between  Fort 
Ord  and  Pensacola.  Figure  8.4  illustrates  the  satisfaction 
index  for  individual  questions  by  combined  work  groups. 
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a.  Question  1   -   Relationship  with   the  Information 

Management    (MID/IMD)   Staff 

As  noted  earlier,  the  level  of  satisfaction  towards 
the  MID/IMD  staff  is  relatively  high.  Pensacola ' s  overall 
level  of  satisfaction  was  higher  than  Fort  Ord's.  Their 
combined  administrative  group  and  combined  ancillary  group 
responses  to  this  guestion  are  practically  the  same.  Overall, 
Pensacola  shows  a  slightly  higher  level  of  satisfaction  for 
their  MID  staff.  There  was  no  statistically  significant 
difference  at  alpha  =  0.05  for  the  combined  work  groups  or 
hospitals. 

b.  Question  2  -  Processing  of  Requests  for  Changes 
to  Existing  Systems 

The  responses  to  this  question  received  the  lowest 
satisfaction  index  rating.  Both  combined  work  groups  and 
hospitals  responses  were  negative  towards  the  processing  of 
requests  for  outside  services.  However,  Fort  Ord  displayed 
a  statistically  significant  lower  difference  lower  at  alpha  = 
0.05. 

c.  Question  3   -  Degree  of  Training  Provided  to  Users 
The  level  of  satisfaction  for  this  item  was  rated 

next  to  the  lowest  by  both  the  combined  work  groups  and 
hospitals.  There  was  no  statistically  significant  difference 
at  alpha  =  0.05. 
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d.  Question  4   -  User's  Understanding  of  the  System 

Fort  Ord  and  the  combined  administrative  group 
displayed  the  highest  level  of  satisfaction  toward 
comprehension  of  their  system.  There  was  no  statistically 
significant  difference  between  at  alpha  =  0.05. 

e.  Question  5  -  User's  Feeling  of  Participation 

The  work  groups  and  hospitals  show  relatively  the 
same  level  of  satisfaction  for  involvement  and  participation 
in  services  and  the  functioning  of  the  system.  There  was  no 
statistically  significant  difference  at  alpha  =  0.05. 

f.  Question  6  -  Attitude  of  the  Information 

Management   (MID/IMD)   Staff 
Fort  Ord  the  combined  administrative  group  tend  to 
show  a  slightly  higher  level  of  satisfaction  toward  their 
MID/IMD  staffs.    There  was  no  statistically  significant 
difference  at  alpha  =  0.05. 

g.  Question  7  -  Reliability  of  Output  Information 
Fort  Ord  perceives  slightly  higher  degree  of 

satisfaction  for  reliability  of  output  information.  The 
combined  work  groups'  level  of  satisfaction  seems  were 
comparable.  There  was  no  statistically  significant  difference 
at  alpha  =  0.05. 


80 


h.    Question   8    -   Relevancy   of  Output    Information    (to 
intended,  function) 

Fort  Ord  and  the  combined  administrative  group  had 
a  higher  level  of  satisfaction  regarding  the  relevancy  of 
output  information.   There  was  no  statistically  significant 
difference  at  alpha  =  0.05. 

i.    Question  9   -  Accuracy  of  Output  Information 

Fort  Ord  and  the  combined  administrative  group  tend 
to  be  more  confident  in  the  accuracy  of  the  output 
information.  There  was  no  statistically  significant 
difference  at  alpha  =  0.05. 

j.    Question  10  -  Precision  of  Output  Information 

The  precision  of  the  output  information  is  more 
favorable  to  Fort  Ord  and  the  combined  administrative  group. 
There  was  no  statistically  significant  difference  at  alpha  = 
0.05. 

k.    Question  11   -  Communication  with  the  Information 
Management  Division   (MID/IMD)   Staff 

The  manner  and  methods  of  information  exchange 
between  the  users  and  MID/IMD  staff  are  highly  valued  by  the 
hospitals  and  the  combined  work  groups.  There  was  no 
statistically  significant  difference  by  t-test  at  alpha  = 
0.05. 
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I.  Question     12     -     Time     Required     for     New     Systems 
Development 

According  to  the  hospitals'  and  combined  work 
groups'  level  of  satisfaction,  the  elapsed  time  for  new 
systems  development  by  DMSSC  or  contractors  is  practically 
unacceptable.   Fort  Ord  may  be  slightly  more  tolerant  than 

Pensacola  for  development  of  new  systems .    There  was  no 
statistically  significant  differences  at  alpha  =  0.05. 
in.  Question  13  -  Completeness  of  Output 

Almost  an  even  level  of  satisfaction  amongst  the 
hospitals  and  the  combined  work  groups.  They  show  some 
satisfaction  towards  the  comprehensiveness  of  the  output 
information.  There  was  no  statistically  significant  difference 
at  alpha  =  0.05. 

2 .   Combined  Grouped  Factors 

Fort  Ord's  and  Pensacola 's  overall  responses  for  the 
grouped  factors  are  joined  together  to  form  overall  combined 
grouped  factors.  These  overall  combined  grouped  factors  are 
broken  down  to  form  overall  work  group  factors.  Refer  to 
chapters  six  and  seven  for  further  discussion  on  group 
factors.  The  following  sections  will  discuss  the  overall 
combined  factors  and  combined  work  group  factors  for  Fort  Ord 
and  Pensacola.  The  physicians  will  not  be  included  in  the 
discussion  compared  by  work  group.  Neither  the  hospitals'  or 
work  groups'  satisfaction  level  for  any  of  the  factors  were 
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statistically  different  at  significance  level  alpha  =  0.05. 
Figure  8.5  represents  the  overall  grouped  factors  for  Fort  Ord 
and  Pensacola. 
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Figure  8.6  represents  the  overall  factors  by  work 


groups 
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a.  Factor  A   (Local  MID/IMD  Staff  and  Services) 

Fort  Ord  and  the  combined  ancillary  group  has  the 
highest  level  of  satisfaction  for  the  MID/IMD  staff  and 
services.   This  factor  was  rated  the  highest  overall.   There 
was  no  statistically  significance  difference  at  alpha  =  0.05. 

b.  Factor  A   (Contractor  Services) 

Consistent  with  other  results,  this  factor  received 
the  lowest  level  of  satisfaction  of  any  factor.  There  was  no 
statistically  significant  difference  at  alpha  =  0.05. 

c.  Factor  B  -  Information  Product 

This  factor  is  the  second  highest  rated  of  the  four 
factors.  There  was  no  statistically  significant  difference  at 
alpha  =  0.05. 

d.  Factor  C  -  Knowledge  and  Involvement 

This  factor  showed  no  statistically  significant 
difference  between  the  level  of  satisfaction  among  the  work 
groups.  The  administrative  work  group  and  Fort  Ord  rated 
knowledge  and  involvement  slightly  better  than  the  other 
groups. 
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IX.     COMPARISON  OF  RESULTS  FROM  SILAS  B.  HAYS  HOSPITAL, 

NAVAL  HOSPITAL,  PENSACOLA  (AQCESS)  AND  NAVAL  HOSPITAL, 
CHARLESTON  (CHCS) 


A.   INTRODUCTION 

This  chapter  compares  AQCESS  and  the  CHCS  user 
satisfaction  findings.  Both  systems  are  fully  described  in 
chapters  II  and  III.  The  survey  results  for  user  satisfaction 
are  compared  with  respect  to  each  site,  system  versus  system 
and  each  work  group.  The  criteria  are  as  follows:  1)  overall 
satisfaction,  2)  responses  to  individual  questions,  3)  three 
group  factors:  MID  staff  and  services;  information  system 
product;  knowledge  or  involvement.  ( Baroudi  and  Orlikowski, 
1988) 

1.  Overall  Satisfaction  (Pensacola,  Fort  Ord,  Charleston) 
The  mean  overall  satisfaction  index  of  the  combined 
sites  is  11.07  on  a  scale  ranging  from  -39  to  +39.  Charleston 
(CHCS  site)  displays  the  highest  overall  satisfaction  at 
11.25,  followed  by  Fort  Ord  at  10.47.  Pensacola  registers  the 
least  overall  satisfaction  at  9.025.  Using  a  t-test,  the 
level  of  satisfaction  was  not  statistically  significant  at 
alpha  =  0.05.  Figure  9.1  illustrates  the  overall  satisfaction 
index  for  each  site. 
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2.   Individual  Questions 

Figure  9.2  displays  the  average  level  of  satisfaction 
for  each  site.  Responses  to  questions  1,  6  and  11  indicate  a 
high  level  of  user  satisfaction.   As  stated  earlier,  these 
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questions  are  concerned  with  the  Management  Information 
(MID/IMD)  staff.  Consistent  with  previous  findings,  responses 
to  questions  2,  3  and  12  stand  out  as  exhibiting  a  relatively 
low  satisfaction  index.  Questions  2  and  12  are  concerned  with 
outside  contractors  and/or  DMSSC  services  and  question  3, 
training.  The  following  discussion  will  further  elaborate  on 
each  question. 

a.  Question  1   -  Relationship  with  the  Management 
Information   (MID/IMD)   Staff 

The  level  of  satisfaction  towards  the  MID  staff  is 
relatively  high  (second  highest  satisfaction  index). 
Charleston  shows  the  highest  level  of  satisfaction  of  the 
three  sites,  closely  followed  by  Pensacola.  Fort  Ord  reports 
the  lowest  satisfaction  index.  There  is  no  statistically 
significant  difference  at  alpha  =  0.05. 

b.  Question  2  -  Processing  of  Requests  for  Changes 
to  Existing  Systems 

All  three  sites  reports  a  negative  level  of 
satisfaction  for  time  to  process  requests  for  changes  to 
existing  system.  This,  the  lowest  scoring  response,  is 
consistent  with  previous  descriptions.  Pensacola  gave  it  the 
lowest  rating,  followed  by  Charleston  and  Fort  Ord, 
respectively.  The  level  of  satisfaction  for  Pensacola  was 
statistically  significant  difference  lower  versus  Charleston's 
level  of  satisfaction  at  alpha  =  0.05. 
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c.  Question  3   -  Degree  of  Training  Provided   to  Users 

The  level  of  satisfaction  of  each  site  was  positive 
but  nearly  neutral  at  Pensacola  and  Fort  Ord.  Charleston 
reported  the  highest  satisfaction  index.  However,  Pensacola 
showed  a  statistically  significant  lower  difference  to 
Charleston's  level  of  satisfaction  at  alpha  =  0.05. 

d.  Question  4  -  User's  Understanding  of  the  System 
Fort  Ord  and  Charleston  were  nearly  equal  in 

satisfaction,  while  Pensacola  reported  a  lower  satisfaction 
index.  There  is  no  statistically  significant  difference  at 
alpha  =0.05. 

e.  Question  5  -  User's  Feeling  of  Participation 
Charleston  rated  this  item  a  little  higher  than 

Fort  Ord  or  Pensacola,  which  reported  a  similar  level  of 
satisfaction.  However,  Pensacola  showed  a  statistically 
significant  lower  difference  than  Charleston's  level  of 
satisfaction  at  alpha  =  0.05. 

f.  Question  6  -  Attitude  of  the  Management 
Information   (MID/IMD)   Staff 

This  was  the  highest  rated  satisfaction  index  of 
all  13  indices.  Charleston's  response  rated  very  high  while 
Pensacola  followed  closely.  Fort  Ord's  response  was  rated 
lower,  but  positive.  Pensacola 's  and  Fort  Ord's  level  of 
satisfaction  show  a  statistically  significant  lower  difference 
to  Charleston's  level  of  satisfaction  at  alpha  =  0.05. 
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g.    Question  7   -  Reliability  of  Output  Information 

Fort  Ord  rated  this  item  higher  than  either 
Pensacola  or  Charleston,  which  responded  similarly.  There  is 
no  statistically  significant  difference  at  alpha  =  0.05. 

h.    Question   8    -   Relevancy  of  Output    Information    (to 

intended  function) 

Again,  Fort  Ord  rated  this  item  higher  than  either 
Pensacola  or  Charleston,  which  responded  similarly.   There  is 
no  statistically  significant  difference  at  alpha  =  0.05. 
i.    Question  9  -  Accuracy  of  Output  Information 

The  response  to  this  question  resulted  in  a 
relatively  high  satisfaction  index  for  Charleston  which  is 
followed  by  Fort  Ord.  Pensacola' s  level  of  satisfaction  shows 
a  statistically  significant  lower  difference  to  Charleston's 
level  of  satisfaction  at  alpha  =  0.05. 

j.    Question  10   -  Precision  of  Output  Information 

Pensacola  and  Charleston  reported  similar  levels  of 
satisfaction.  Fort  Ord,  although  not  to  any  large  degree, 
reported  a  higher  level  of  satisfaction.  There  is  no 
statistically  significant  difference  at  alpha  =  0.05. 

k.  Question    11    -    Communication    with    the    Management 

Information   (MID/IMD)   Staff 

This  item  is  rated  among  the  top  three  in  levels  of 
satisfaction.  Charleston  has  the  highest  index,  Fort  Ord  is 
second  and  Pensacola,  although  positive,  reports  the  lowest 
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level  of  satisfaction.   There  is  no  statistically  significant 
difference  at  alpha  =  0.05. 

I.  Question     12     -     Time     Required     for     New     Systems 
Development 

This  item  is  rated  as  the  second  lowest  among  the 

13  responses.   Charleston's  response  results  in  a  negative 

index  which  has  statistically  significant  lower  difference  at 

alpha  =  0.05.   Fort  Ord  has  the  highest  level  of  satisfaction. 

m.    Question  13   -  Completeness  of  Output 

The  responses  to  this  question  are  relatively 
close.     Charleston   demonstrates   the   lowest   level   of 
satisfaction  while  Fort  Ord  has  the  highest.   There  is  no 
statistically  significant  difference  at  alpha  =  0.05. 
3.   Grouped  Factors 

The  following  sections  will  discuss  the  overall 
combined  factors  by  sites.  Fort  Ord's  and  Pensacola's  levels 
of  satisfaction  displays  a  statistically  significant  lower 
difference  to  Charleston's  level  of  satisfaction  at  alpha  = 
0.05.  Figure  9.3  represents  each  site's  level  of  satisfaction 
with  respect  to  the  grouped  factors. 

a.  Factor  A   (Local  MID/IMD  Staff  and  Services) 

As  seen  in  Figure  9.3,  Charleston  displays  the 
highest  level  of  satisfaction,  followed  by  Fort  Ord.   The 
lowest  level  of  satisfaction  is  reported  by  Pensacola.   Both 
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Figure  9 . 3 

Fort  Ord ' s  and  Pensacola's  levels  of  satisfaction  results  in 
a  statistically  significant  lower  difference  to  Charleston's 
at  alpha  =  0.05. 

b.  Factor  A   (Contractor,   DMSSC) 

This  factor  received  the  lowest  level  of 
satisfaction  of  any  group  factor.  All  sites  rated  this  factor 
extremely  low,  with  Pensacola  and  Fort  Ord  almost  rating  it 
nearly  neutral.  There  is  no  statistically  significant 
difference  at  alpha  =  0.05. 

c.  Factor  B  -   Information  System  Product 

This  was  the  second  highest  rated  factor.  All 
three  sites  report  similarly,  with  Fort  Ord  recording  the 
highest  level  of  satisfaction.  There  is  no  statistically 
significant  difference  at  alpha  =  0.05. 
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d.    Factor  C  -  Knowledge  and  Involvement 

The  second  lowest  rated  factor,  all  sites  report 
similarly.  Charleston's  level  of  satisfaction  is  the  highest, 
followed  by  Ford  Ord's.  Pensacola's  level  of  satisfaction  is 
the  lowest.  There  is  no  statistically  significant  difference 
at  alpha  =  0.05. 

4.  Overall  Satisfaction  (CHCS  vs  AQCESS) 

The  average  overall  satisfaction  index  of  CHCS  and 
AQCESS  is  10.54.  CHCS  reports  the  highest  overall 
satisfaction  (11.25),  with  AQCESS  at  10.13.  Using  a  t-test, 
the  difference  in  level  of  satisfaction  is  not  statistically 
significant  at  alpha  =  0.05.  Figure  9.4  illustrates  the 
overall  satisfaction  index  for  each  system. 

5 .  Individual  Questions 

Figure  9.5  displays  the  average  level  of  satisfaction 
for  each  site.  Again,  responses  to  questions  1,  6  and  11 
indicate  a  high  level  of  user  satisfaction.  (MID/IMD)  Also 
consistent  with  previous  findings,  responses  to  questions  2, 
3  and  12  stand  out  as  exhibiting  a  relatively  low  satisfaction 
index.  Questions  2  and  12  are  concerned  with  outside 
contractors  and/or  DMSSC  services  and  question  3,  training. 
The  following  discussion  will  further  examine  each  question. 
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a.  Question  1   -  Relationship  with  the  Management 
information   (MID/IMD)   Staff 

The  level  of  satisfaction  towards  the  MID  staff  is 
relatively  high  (second  highest  satisfaction  index).  CHCS 
users  have  the  higher  level  of  satisfaction,  however,  there  is 
no  statistically  significant  difference  at  alpha  =  0.05 
between  the  two  indices. 

b.  Question  2  -  Processing  of  Requests  for  Changes 
to  Existing  Systems 

This,  the  lowest  scoring  response,  received  a 
negative  index  from  both  system  users.  CHCS  users  reported 
the  lowest  satisfaction  rating,  however,  there  was  no 
statistically  significant  difference  at  alpha  =  0.05. 

c.  Question  3  -  Degree  of  Training  Provided  to  Users 
The  level  of  satisfaction  for  each  system  was 

positive  but  nearly  neutral  for  AQCESS  users.   CHCS  showed  a 
statistically  significant  lower  difference  at  alpha  =  0.05. 

d.  Question  4  -  User's  Understanding  of  the  System 
CHCS  was  rated  higher  in  satisfaction  than  AQCESS 

for   this   item.     There  was,   however,   no   statistically 
significant  difference  between  the  indices  at  alpha  =  0.05. 

e.  Question  5  -  User's  Feeling  of  Participation 
CHCS  users  reported  a  higher  index  than  AQCESS 

users.    AQCESS  showed  a  statistically  significant  lower 
difference  at  alpha  =  0.05. 
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f.  Question   6  -  Attitude  of  the  Management 
Information    (MID/IMD)    Staff 

Again,  this  was  the  highest  rated  satisfaction 
index  of  all  13  indices.  CHCS  users'  response  was  rated  very 
high  while  AQCESS  users,  who  yielded  a  positive  rating,  were 
statistically  significant  difference  lower  at  alpha  =  0.05. 

g.  Question  7  -  Reliability  of  Output   Information 
The   satisfaction   indices   for   this   item   were 

positive  and  virtually  the  same. 

h.    Question   8    -    Relevancy   of  Output    Information    (to 
intended  function) 

Both  CHCS  and  AQCESS  users  gave  positive  responses. 
There  was  no  statistically  significant  difference  at  alpha  = 
0.05. 

i.    Question  9  -  Accuracy  of  Output  Information 

The  response  to  this  question  resulted  in  a 
relatively  high  satisfaction  index  for  CHCS  users.  However, 
AQCESS  users  show  a  statistically  significant  difference  lower 
at  alpha  =  0.05. 

j.  Question  10  -  Precision  of  Output  Information 

The  CHCS  and  AQCESS  users  responded  similarly  with 
no  statistically  significant  difference  at  alpha  =  0.05. 
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k.    Question    11    -    Communication    with    the    Management 
Information   (MID/IMD)   Staff 

Rated  third  highest  index  overall,  there  was  no 
statistically  significant  difference  at  alpha  =  0.05. 

2.  Question     12     -     Time     Required     for     New     Systems 
Development 

This  item  is  rated  as  the  second  lowest  among  the 
13  indices.  CHCS  users'  response  resulted  in  a  negative 
index,  while  AQCESS  users  showed  a  statistically  significant 
higher  difference  at  alpha  =  0.05. 

m.  Question  13  -  Completeness  of  Output 

The  responses  to  this  question  were  positive  with 
no  statistically  significant  difference  at  alpha  =  0.05. 
6.   Grouped  Factors 

The  following  sections  will  discuss  the  overall 
combined  factors  for  the  respondents  of  the  CHCS  and  AQCESS 
systems.  Figure  9.6  represents  each  site's  level  of 
satisfaction  in  relation  to  group  factors. 

a.    Factor  A   (Local  MID/IMD  Staff  and  Services) 

As  seen  in  Figure  9.6,  CHCS  respondents  report  a 
much  higher  level  of  satisfaction  for  their  local  staff  and 
services.   AQCESS  respondents'  level  of  satisfaction  show  a 
statistically  significant  lower  difference  at  alpha  =  0.05. 
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Figure   9.6 

b.  Factor  A   (Contractor,    dmsso 

This  factor  received  the  lowest  level  of 
satisfaction  of  any  group  factor.  Both  systems'  users  rate 
this  factor  low,  with  CHCS  being  statistically  significant 
difference  lower  at  alpha  =  0.05. 

c.  Factor  B   -   Information  System  Product 

CHCS  and  AQCESS  users  responded  essentially  the 
same.   There  is  no  statistically  significant  difference  in 
levels  of  satisfaction  from  either  systems'  respondents  at 
alpha  =  0.05. 

d.  Factor  C  -  Knowledge  and  Involvement 

The  users  of  the  CHCS  system  display  a  slightly 
higher  level  of  satisfaction  than  the  AQCESS  users.  There  is 
no  statistically  significant  difference  at  alpha  =  0.05. 
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X.   COMBINED  DESCRIPTIVE  FINDINGS  OF  CHCS  AND  AQCESS 


A.   INTRODUCTION 

The  AQCESS  and  CHCS  systems,  although  similar  in  many 
areas,  are  functionally  different  systems.  However,  the 
survey  responses  from  both  systems  are  joined  to  assist  in  the 
analysis  of  findings  across  these  systems.  Chapter  IX 
compared  the  Charleston  Naval  Hospital's  CHCS  system  to 
Pensacola  Naval  Hospital's  and  Fort  Ord  Army  Hospital's  AQCESS 
systems.  This  chapter  will  examine  the  combined  results  of 
the  survey  responses  from  the  hospitals.  Figure  10.1  shows  a 
breakdown  of  the  respondents  by  work  groups  from  the  combined 
hospitals. 
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B.   OVERALL  USER  SATISFACTION  FOR  THE  COMBINED  INFORMATION 
SYSTEMS 

The  overall  satisfaction  is  measured  by  the  mean  of  the 
sum  of  the  responses  to  the  survey  questions  from  the 
hospitals.  A  t-test  the  overall  satisfaction  level  of  the 
ancillary  group  and  administrative  group  has  no  statistically 
significant  difference  at  alpha  =  0.05.  The  physicians  are 
statistically  significant  difference  lower  response  at  alpha 
=  0.05.  The  survey  shows  that  the  ancillary  group  displays 
the  highest  overall  satisfaction  index  at  11.47.  When 
compared  to  the  overall  satisfaction  index  of  the 
administrative  groups  level  at  11.43,  the  ancillary  group  and 
administrative  group  overall  satisfaction  for  their 
information  systems  are  approximately  equal.  The  physicians 
level  of  satisfaction  index  for  the  systems  are  the  lowest  at 
1.35.  The  overall  satisfaction  level  was  10.14.  Figure  10.2 
illustrates  the  combined  work  groups  level  of  satisfaction  for 
the  AQCESS  and  CHCS  hospital  information  systems. 

1 .   Individual  Question  Responses  for  AQCESS  and  CHCS 

The  method  used  to  calculate  the  values  of  the 
individual  question  responses  is  described  in  earlier 
chapters.  Together,  the  CHCS  and  AQCESS  systems  show  the  same 
trend  as  the  other  comparisons.  Questions  1,  6  and  11  (deals 
with  issues  concerning  MID/IMD)  show  the  highest  level  of 
satisfaction.   Consistent,  questions  2,  3  and  12  (deals  with 
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DMSSC  services  and  management)  have  the  lowest  rated  indices. 
Figure  10.3  displays  the  overall  level  of  satisfaction  for 
individual  questions  for  the  CHCS  and  AQCESS  combined  work 
groups. 
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In  the  comparisons  of  one  work  group  versus  the  remaining 
work  groups,  the  patterns  of  satisfaction  levels  for 
individual  question  responses  are  similar.  Questions  1,  6  and 
11  consistently  rate  the  highest  level  of  satisfaction,  while 
questions  2,  3  and  12  consistently  rate  the  lowest  level  of 
satisfaction  by  the  respondents  of  the  combined  work  groups. 
Each  question  will  be  discussed  further  in  the  following 
sections.  Figure  10.4  displays  the  responses  to  individual  by 
physicians  compared  to  the  remainder  of  respondents;  Figure 
10.5  ancillary  compared  to  the  other  groups;  Figure  10.6 
administrative  compared  to  the  other  groups. 
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a.  Question  1   -  Relationship  with  the  Information 

Management    (MID/IMD)   Staff 

As  noted  earlier,  the  level  of  satisfaction  towards 
the  MID/IMD  staff  is  relatively  high.  The  level  of 
satisfaction  is  higher  for  the  ancillary  group  than  the  others 
but  not  much  higher  than  the  administrative  group.  The 
physicians  show  a  statistically  significant  lower  difference 
at  alpha  =  0.05. 

b.  Question  2  -  Processing  of  Requests  for  Changes 
to  Existing  Systems 

The  responses  to  this  question  received  the  lowest 
satisfaction  index  rating.  All  work  groups'  responses  were 
negative  towards  the  processing  of  requests  for  outside 
services  to  make  changes  to  the  existing  systems.  However, 
the  level  of  satisfaction  of  the  administrative  work  group 
showed  a  statistically  significant  higher  difference  at  alpha 
=  0.05.  The  ancillary  group  was  much  lower  followed  by  the 
physicians. 

c.  Question  3  -  Degree  of   Training  Provided  to  Users 
The  ancillary  group  shows  a  higher  level  of 

satisfaction  for  the  amount  of  training  they  have  received  on 
using.     The  physicians  expressed   the   lowest   level   of 
satisfaction  for  the  degree  of  training  they  have  received. 
The  level  of  satisfaction  for  the  ancillary  group  shows  a 
statistically  significant  higher  difference  and  the  physicians  ' 
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level  of  satisfaction  has  a  statistically  significant  lower 
difference  lower  at  alpha  =  0.05. 

d.  Question  4  -  User's  Understanding  of  the  System 
The  ancillary  group  displayed  the  highest  level  of 

satisfaction  toward  comprehension  of  their  systems.  The  level 
of  satisfaction  for  the  ancillary  group  has  a  statistically 
significant  higher  difference  at  alpha  =  0.05.  The  level  of 
satisfaction  for  the  physicians  showed  a  statistically 
significant  lower  difference  at  alpha  =  0.05. 

e.  Question  5  -  User's  Feeling  of  Participation 

The  ancillary  group  showed  the  highest  level  of 
satisfaction  for  involvement  and  participation  in  services  and 
the  functioning  of  the  system  followed  closely  by  the 
administrative  group.  The  ancillary  group's  level  of 
satisfaction  showed  a  statistically  significant  higher 
difference  while  the  physicians'  level  of  satisfaction  showed 
a  statistically  significant  lower  difference  at  alpha  =  0.05. 

f.  Question  6  -  Attitude  of  the  Information 

Management  (MID/IMD)  Staff 
Even  though  the  physicians  showed  level  of 
satisfaction  that  was  a  statistically  significant  lower 
difference  at  alpha  =  0.05,  all  work  groups  responded 
favorably  to  the  attitudes  of  the  MID/IMD  staffs.  The 
ancillary  groups'  level  of  satisfaction  was  the  highest, 
followed  closely  by  the  administrative  group. 
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g.    Question  7   -  Reliability  of  Output  Information 

The  perceptions  of  the  work  groups  for  reliability 
of  output  information  were  relatively  low.  The  administrative 
group's  level  of  satisfaction  was  slightly  higher  than  the 
ancillary  work  group's.  The  physicians*  level  of  satisfaction 
was  extremely  low  and  showed  a  statistically  significant  lower 
difference  at  alpha  =  0.05. 

h.    Question   8    -   Relevancy   of  Output   Information    ( to 
intended  function) 

Regarding  the  relevancy  of  output  information,  the 
administrative  group  had  a  higher  level  of  satisfaction, 
followed  closely  by  the  ancillary  group.  The  physicians' 
level  of  satisfaction  showed  a  statistically  significant  lower 
difference  at  alpha  =  0.05. 

i.    Question  9  -  Accuracy  of  Output  Information 

All  work  groups  showed  positive  degree  of 
confidence  in  the  accuracy  of  the  output  information.  The 
level  of  satisfaction  for  the  ancillary  group  is  slightly 
higher  but  nearly  even  with  the  satisfaction  level  of  the 
administrative  group.  The  physicians  level  of  satisfaction  is 
the  lowest.  There  was  no  statistically  significant  difference 
in  the  level  of  satisfaction  for  the  groups. 

j.    Question  10  -  Precision  of  Output  Information 

All  work  groups'  level  of  satisfaction  for  the 
precision  of  the  output  information  was  relatively  low.   The 
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administrative  group's  level  of  satisfaction  was  the  highest, 
followed  closely  by  the  ancillary  group.  The  physicians' 
level  of  satisfaction  is  the  lowest.  There  was  no 
statistically  significant  difference  in  level  of  satisfaction 
at  alpha  =  0.05. 

k.    Question   11    -    Communication  with    the   Information 

Management  Division   (MIS/IMD) 

The  manner  and  methods  of  information  exchange 
between  the  users  and  the  MID/IMD  staffs  was  highly  valued  by 
the  work  groups.  The  ancillary  group's  level  of  satisfaction 
was  the  highest.  Slightly  closer  were  the  administrative 
group  and  physicians.  This  was  the  highest  level  of 
satisfaction  recorded  by  the  physicians.  There  was  no 
statistically  significant  difference  at  alpha  =  0.05. 
2.  Question  12  -  Time  Required  for  New  Systems 

The  work  groups'  level  of  satisfaction,  for  time 
required  to  develop  new  systems  by  DMSSC  or  contractors,  was 
very  low.  The  administrative  group's  level  of  satisfaction 
for  the  systems  was  the  highest,  followed  by  the  ancillary 
group.  The  physicians'  level  of  satisfaction  was  extremely 
low.  The  administrative  group's  level  of  satisfaction 
revealed  a  statistically  significant  higher  difference  and  the 
physicians'  level  of  satisfaction  revealed  a  statistically 
significant  lower  difference  at  alpha  =  0.05. 
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m.  Question  13   -  Completeness  of  Output 

The  work  groups  showed  some  satisfaction  towards 
the  comprehensiveness  of  the  output  information.  The 
administrative  group's  level  of  satisfaction  was  the  highest, 
followed  by  the  ancillary  group.  The  physicians  recorded  the 
lowest  level  of  satisfaction.  The  administrative  group's 
level  of  satisfaction  showed  a  statistically  significant 
higher  difference  at  alpha  =  0.05. 
2 .   Combined  Grouped  Factors 

The  following  sections  will  discuss  the  overall 
combined  factors  by  work  groups.  The  physicians'  level  of 
satisfaction  displayed  a  statistically  significant  lower 
different  for  the  four  factors  at  alpha  =  0.05.  Figure  10.7 
represents  the  work  groups'  level  of  satisfaction  with  respect 
to  the  sub-total  factors.  Figure  10.8  displays  the 
satisfaction  level  for  the  factors  by  the  physicians  compared 
to  the  remainder  of  the  groups;  Figure  10.9  ancillary  compared 
to  the  others;  Figure  10.10  administrative  compared  to  the 
others. 

a.  Factor  A   (Local  MID/IMD  Staff  and  Services) 

The  ancillary  group  had  the  highest  level  of 
satisfaction  for  the  MID/IMD  staff,  followed  closely  by  the 
administrative  group.  Although  this  factor  was  rated  as  being 
the  highest  level  of  satisfaction  by  the  work  groups,  the 
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physicians'   level  of  satisfaction  showed  a  statistically 
significant  lower  difference  at  alpha  =  0.05. 
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Jb.  Factor  A   (Contractor  Services) 

The  type  and  quality  of  services  rendered  by  DMSSC 
or  contractors  have  been  consistently  rated  low  by  the  work 
groups  in  all  comparisons.  The  ancillary  group  showed  the 
only  positive  level  of  satisfaction  amongst  the  groups,  but 
was  nearly  neutral.  The  administrative  group  followed  with  a 
negative  rating  for  level  of  satisfaction  and  the  physician 
with  the  lowest.  The  physicians'  level  of  satisfaction  showed 
a  statistically  significant  lower  difference  at  alpha  =  0.05. 

c.  Factor  B  -  Information  Product 

This  factor  was  rated  positive  by  all  groups.  The 
highest  level  of  satisfaction  was  given  by  the  ancillary  group 
which  was  comparable  to  the  administrative  group's.  The 
physicians'  level  of  satisfaction  for  the  information  product 
was  the  lowest  and  showed  a  statistically  significant  lower 
difference  at  alpha  =  0.05. 

d.  Factor  C  -  Knowledge  and  Involvement 

The  administrative  group's  level  of  satisfaction 
for  knowledge  and  involvement  was  the  highest,  closely 
followed  by  the  ancillary  group.  The  physicians  showed  a 
statistically  significant  lower  difference,  at  alpha  =  0.05. 
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XI.   ANALYSIS  OF  FINDINGS 


A.   INTRODUCTION 

Performance  evaluation  is  a  crucial  part  of  the  management 
of  an  information  system.  The  amounts  of  resources  invested 
and  the  organization's  dependence  on  information  processing 
warrant  the  most  efficient,  effective  use  of  the  information 
system.  However,  without  direction  or  objective,  this 
evaluation  is  simply  an  assessment  of  the  current  operation  of 
the  information  system.  (Mensching  and  Adams,  1991)  The 
purpose  of  this  research  is  to  measure  and  document  the  user 
information  satisfaction  of  the  Automated  Quality  of  Care 
Evaluation  Support  System  ( AQCESS )  at  the  Naval  Hospital, 
Pensacola,  Florida  and  Silas  B.  Hays  Hospital,  Fort  Ord, 
California.  These  findings  from  the  AQCESS  system  will  be 
compared  to  those  from  the  Composite  Health  Care  System  ( CHCS ) 
at  the  Naval  Hospital,  Charleston,  South  Carolina  (Hurd, 
1991).  The  results  should  not  be  viewed  as  being  a  conclusive 
evaluation  of  the  AQCESS  or  CHCS  systems,  but  provide  a 
benchmark  for  analyzing  user  satisfaction  and  identifying 
possible  areas  of  discontent.  To  obtain  a  deeper 
understanding  of  the  identified  and  highlighted  symptoms  of 
discontent,  interviews  should  be  conducted  to  examine  the 
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development  and  operating  procedures  of  the  particular  work 
group  involved  in  order  to  determine  the  underlying  causes  by 
adding  context  and  history.  Upon  completion  of  this  process, 
it  may  become  possible  to  make  refinements  and  tuning  to  the 
system  to  increase  its  effectiveness  and,  perhaps  secondarily, 
its  efficiency. 

The  first  section  will  analyze  the  survey  findings  in 
chapters  VI.  through  X.  It  will  primarily  focus  on  the 
differences  and  similarities  among:  work  groups 
(administrative,  ancillary,  and  physicians)  at  Pensacola;  work 
groups  at  Fort  Ord;  combined  work  groups  of  Pensacola  and  Fort 
Ord;  CHCS  and  AQCESS  systems  at  Charleston,  Pensacola  and  Fort 
Ord;  combined  work  groups  of  Pensacola,  Fort  Ord  and 
Charleston. 

B.   PENSACOLA  OVERALL  SATISFACTION 

No  one  can  deny  that  the  AQCESS  systems  have  not  been  a 
benefit  to  those  sites  that  have  this  system.  Automation  has 
improved  their  overall  efficiency  and  effectiveness.  (NMDSC, 
1991).  Pensacola  Naval  Hospital  has  benefitted  tremendously 
from  the  AQCESS  system.  In  order  for  the  AQCESS  system  to 
continue  to  be  a  successful  computer  system,  it  must  provide 
the  needed  information  required  by  the  users  (Olson  and 
Baroudi,  1983).  The  following  discussion  will  attempt  to 
analyze  the  levels  of  dissatisfaction  that  are  statistically 
significant  by  the  respondents  from  Pensacola.   Although  not 
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conclusive,  it  should  be  used  as  a  baseline  measure  to  help 
verify  areas  in  the  AQCESS  system  that  may  require 
improvements  or  fine  tuning. 

1 .  Combined  Groups 

The  overall  satisfaction  index  is  a  general 
description  of  the  user  satisfaction  with  the  system.  The 
survey  average  satisfaction  index  is  9.64.  The  respondents 
average  overall  satisfaction  is  characterized  as  only  being 
marginally  satisfied. 

2.  Comparison  of  Work  Groups 

The  physicians'  level  of  satisfaction  showed  a 
statistically  significant  lower  difference  than  the 
administrative  and  ancillary  work  groups.  However,  the 
administrative  group  was  somewhat  more  satisfied  than  the 
ancillary  group,  but  only  slightly. 

The  physicians'  group  was  dissatisfied  with  the 
system,  which  is  indicated  by  their  low  average  satisfaction 
index  at  -4.16.  Generally,  the  only  way  the  physicians 
legitimately  interface  with  the  system  is  by  reviewing 
statistics  of  their  performance,  which  are  compiled  by  the 
Quality  Assurance  personnel.  To  a  lesser  degree,  the 
physicians  interact  with  the  system  through  the  Ad  Hoc  Report 
Generator  module  in  the  system.  The  physicians  mainly  view 
the  system  as  a  means  of  reporting  their  performance  of 
patient  care,  which,  according  to  their  perspective,  may  not 
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be  accurate.  Therefore,  any  inaccuracies  in  a  physician's 
performance  report  is  viewed  negatively  by  that  physician  and 
potentially  leads  to  conflict  with  those  who  generate  the 
reports.  So,  it  is  reasonable  to  assume  that  their  overall 
low  level  of  satisfaction  can  be  attributed  to  their  limited 
interface  with  the  AQCESS  system. 

Ancillary  made  up  39  percent  of  the  personnel  surveyed. 
Their  average  satisfaction  index  is  11.49,  slightly  lower 
than  the  administrative  group.  Most  of  the  users  in  this  work 
group  are  technicians  who  use  the  system  for  direct  medical 
care  (e.g.,  canceling  and  editing  disposition  data,  patient 
history,  diagnosis  and  procedure  data  record  tracking  and 
reporting) .  As  the  large  number  of  items  processed  each  day 
in  these  areas  increase,  the  response  time  for  processing 
decreases  to  unacceptable  levels.  It  has  become  routine  for 
ancillary  personnel  to  wait  more  than  five  minutes  to  retrieve 
records.  When  time  is  critical  for  patient  care,  this  waiting 
is  a  source  of  frustration  for  ancillary  personnel. 

The  administrative  group  recorded  the  highest  overall 
satisfaction  score.  Administrative  personnel  use  the  system 
for  purposes  other  than  direct  conduct  of  patient  care.  With 
a  few  exceptions,  the  work  performed  on  the  system  can  be 
accomplished  without  the  pressure  of  a  limited  and  specific 
time  constraints.  Therefore,  they  can  use  the  computer  system 
at  a  comparatively  slower  pace  than  the  other  work  groups. 
Consequently,  interviews  with  most  administrative  personnel 
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revealed  concern  about  the  slow  computer  response  time. 
Additionally,  interviews  revealed  a  lack  of  trust  for  the 
integrity  of  the  output  information  by  the  system.  Some 
administrative  personnel  argued  that  their  inability  to 
correct  data,  once  it  had  been  entered  into  the  system,  made 
the  information  suspect. 

3.   Significant  Individual  Question  Differences  and 

This  section  will  discuss  only  those  responses  to  the 
questions  with  significant  differences  among  the  work  groups. 
The  physicians'  level  of  satisfaction  for  all  questions  showed 
statistically  significant  lower  differences  than  those  of  the 
other  work  groups.  A  low  level  of  satisfaction  for  all 
questions  by  the  physicians  can  be  expected  due  to  their  lack 
of  involvement  and  understanding  of  the  system  coupled  with 
their  limited  role  as  a  user  of  the  system.  Those  questions 
that  are  statistically  significant  for  the  ancillary  and 
administrative  personnel  will  be  discussed  in  the  following 
sections.  Since  the  physicians'  responses  to  the  questions 
are  statistically  significant  in  every  case,  only  those 
questions,  which  will  be  helpful  in  gaining  a  better 
appreciation  of  the  degree  satisfaction,  will  be  analyzed. 
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a.  Relationship   with   the   Management   Information 
Department  (MID) 

The  results  of  the  survey  showed  the  physicians  to 
be  the  least  satisfied  with  the  services  of  by  the  MID  staff. 
Ironically,  their  level  of  satisfaction  for  this  question  was 
the  highest  of  all  their  responses.  Typically,  unless  it  is 
to  report  inaccuracies  in  their  performance  report,  which  is 
generated  by  the  system,  the  physicians'  seldom  communicate 
with  the  MID  staff.  If  this  was  the  only  method  in  which  the 
physicians  and  the  MID  staff  communicated,  then  the 
physicians'  relatively  low  level  of  satisfaction  with  the 
staff  can  be  expected. 

b.  Processing  of  Requests  for  Changes  to  the  Existing 
System 

DMSSC  is  responsible  for  providing  the  support  and 
services  for  the  AQCESS  system.  Request  for  changes  must  go 
to  DMSSC  for  approval  and  funding.  If  enough  AQCESS  sites 
request  or  support  the  change,  then  an  upgrade  to  the  system 
will  be  made.  This  upgrade,  after  testing,  is  then  released 
as  an  improved  version  of  the  previous  AQCESS  system.  The 
latest  version  for  testing  is  AQCESS  7.0,  which  will  take  a 
year  to  release,  if  successful.  (ADPAQCESS  NMDSC,  1991) 
Because  AQCESS  is  under  centralized  control  (DMSSC) ,  changes 
to  AQCESS  will  not  occur  rapidly. 
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c.  User's  Understanding  of  the  System 

Users  with  better  computer  background  and  training 
are  likely  to  be  more  confident  in  their  ability  to  use  the 
system  and  are  more  satisfied  with  the  experience.  The  work 
groups  that  reported  a  low  level  of  satisfaction  in  user 
training  (physicians  in  particular)  demonstrated  a 
significantly  lower  level  of  satisfaction  with  their 
understanding  of  the  system. 

d.  User's  Feeling  of  Involvement 

The  presumption  is  that  user  involvement  will  lead 
to  better  understanding  of  the  system  and  develop  a  system 
tailored  to  meet  specific  needs.  Therefore,  users  will  be 
more  satisfied  with  the  system  than  if  they  had  not  been 
involved.  (Baroudi,  Olson  and  Ives,  1986)  Physicians  are 
rarely  involved  with  the  system.  So,  their  low  of  level 
satisfaction  is  expected.  Consequently,  administrative 
personnel  are  involved  daily  in  the  operations  of  the  system 
and  they  feel  involved  with  the  system. 

e.  Relevancy  of  Output  Information 

When  the  output  information  is  less  than  what  the 
user  perceives,  the  user  will  be  less  satisfied  with  the 
system  ( Conrath  and  Mignen,  1990).  All  groups  have  a  low 
perception  of  the  output  information. 
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f.  Accuracy  of  Output  Information 

Physicians  often  question  the  accuracy  of  their 
statistical  performance  reports.  If  the  information  is 
inaccurate,  then  the  system  may  be  more  likely  to  be  blamed 
for  the  mistake  than  the  person  who  inputs  the  information. 
This  may  be  the  main  source  of  discontent  for  the  physicians. 

g.  Completeness  of  Output 

The  comprehensiveness  of  the  output  information 
relates  to  the  relevancy  and  accuracy  of  the  information.  If 
the  user  perceives  the  information  to  be  lacking,  the  user 
will  more  likely  think  the  information  is  inaccurate  and/or 
irrelevant.  It  is  not  surprising  to  find  the  administrative 
group  scoring  significantly  higher  than  the  other  groups  on 
this  question.  The  administrative  group  does  not  use  this 
information  directly. 

4.   Grouped  factors 

The  overall  management  of  the  AQCESS  system  is 
centrally  controlled  by  DMSSC  who  are  responsible  for  the 
software  support  for  the  system.  Local  support  services  are 
provided  by  the  local  MID.  Therefore,  the  single  factor,  MID 
staff  and  services,  found  in  the  original  study  by  Ives, 
Olson,  and  Baroudi  (1988),  was  altered  to  make  two  separate 
factors:  local  MID  staff  and  services  and  Contractor's 
services. 
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The  physicians  scored  the  four  factors  statistically 
significant  lower  and  the  administrative  group  scored  the  four 
factors  statistically  significant  higher  for  level  of 
satisfaction.  As  alluded  to  earlier,  due  to  the  physicians' 
limited  role  and  involvement  with  the  system,  their  low  level 
of  satisfaction  is  predictable.  The  administrative  group  is 
not  as  constrained  for  time  as  the  ancillary  group.  They  are 
more  involved  with  the  system  and  may  have  a  higher  level  of 
satisfaction,  but  not  significant. 

It  is  noteworthy  to  examine  the  significant  positive 
level  of  satisfaction  towards  the  local  MID  staff  and 
services.  Although  there  was  a  statistically  significant 
lower  difference,  the  physicians'  level  of  satisfaction  for 
this  factor  was  positive,  the  only  factor  to  be  scored 
positive  by  the  physicians.  The  administrative  and  ancillary 
groups'  satisfaction  index  indicated  that  they  were  very 
satisfied  with  the  services  provided  by  the  local  MID  staff. 
Common  sense  would  suggest  that  the  level  of  satisfaction  is 
directly  related  to  effective  communication  ( Conrath  and 
Mignen,  1990).  Thus,  it  can  be  reasonable  to  assume  that  the 
MID  staff  is  effectively  communicating  with  the  ancillary  and 
administrative  group  and,  to  a  lesser  degree,  the  physicians. 

Bailey  and  Pearson  (1983)  listed  contractor  services 
and  time  reguired  for  system  changes  as  the  two  most  freguent 
elements  for  causing  dissatisfaction  among  users.  The  outside 
services  rendered  by  DMSSC  were  viewed  unfavorably  by  the  work 
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groups.  DMSSC  oversees  166  sites  DoD  wide.  Each  site  has  an 
AQCESS  Customer  Support  Representative  who  is  responsible  for 
providing  direct  support  of  that  user.  A  site  has  several 
means  of  reporting  their  problems;  E-mail,  System  Change 
Requests  ( SCRs ) ,  work  shops  etc.  (NMDSC,  1991)  Any  requests 
for  changes  to  a  system  or  its  software,  unless  the  change  is 
critical,  will  be  a  lengthy  process.  Since  most  of  the  users 
are  not  familiar  with  the  centralized  process  or  DMSSC,  it  is 
only  reasonable  to  expect  the  low  level  of  satisfaction  for 
this  factor. 

It  appears  that  the  users  will  be  satisfied  if  the 
output  information  is  accurate,  comprehensive,  reliable  and 
relevant,  the  users'  will  be  satisfied.  The  work  group  was 
slightly  satisfied  with  the  factor  regarding  the  quality  of 
output  information  delivered  by  the  AQCESS  system. 

Powers  and  Dickson  (1973)  argues  that  no  matter  how 
effective  a  system  might  be,  if  it  is  not  perceived  to  be 
satisfactory,  it  will  be  underutilized.  AQCESS  is  training 
intensive  and  is  not  designed  for  the  computer  novice. 
Consequently,  some  of  the  features  of  the  system  are 
underutilized.  (NMDSC,  1991)  If  that  is  the  case,  then  a  low 
level  of  satisfaction  will  be  displayed  by  the  users.  The 
administrative  and  ancillary  groups  displayed  a  nearly  neutral 
satisfaction  index  for  the  knowledge  and  involvement  factor. 
The  satisfaction  index  was  negative  for  the  physicians.  The 
administrative  group  exhibited  a  satisfaction  level  which 
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showed  a  statistically  significant  higher  difference  than  the 
other  two  groups.  The  administrative  group  can  be  considered 
slightly  satisfied  with  this  factor. 

5.   Satisfaction  versus  Time  of  System  Use 

In  a  study  conducted  by  Igbaria  and  Nachman  (1990),  a 
significant  relationship  was  found  between  user  satisfaction 
and  system  utilization.  This  study  showed  an  increase  in 
overall  satisfaction  as  time  of  system  use  increased.  Then, 
the  level  of  satisfaction  decreases  after  extended  use 
(greater  than  11  months)  of  the  system. 

Initially,  new  personnel  are  preoccupied  with  learning 
the  AQCESS  system  in  order  to  learn  their  job.  Once  they  have 
become  comfortable  and  familiar  with  the  AQCESS  system,  the 
users  are  more  concerned  with  doing  their  job.  Conseguently, 
the  users  become  more  intolerant  to  system  down  time  or  slow 
response  time.  Also,  as  their  knowledge  of  and  experience 
with  the  system  grows,  they  can  more  easily  comprehend  the 
reasons  for  system  problems  and  only  wonder  why  the  experts 
can  not  fix  them.  Thus,  their  level  of  satisfaction  for  the 
system  has  decreased  with  time. 

C.   FORT  ORD  OVERALL  SATISFACTION 

A  comparison  of  the  satisfaction  levels  of  the  respondents 
from  the  Silas  B.  Hays  Hospital  will  provide  a  baseline 
measure  for  future  evaluations  of  the  system. 
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1 .  Combined  Groups 

Fort  Ord ' s  average  satisfaction  index  is  10.47.  By 
referring  to  the  index  ranges  of  -39  to  39,  the  respondents' 
average  overall  satisfaction  is  classified  as  being  only 
slightly  satisfied. 

2.  Comparison  of  Work  Groups 

There  were  two  work  groups  identified  in  this  study, 
administrative  and  ancillary,  instead  of  three  because  no  Fort 
Ord  physicians  responded  to  the  survey.  (A  spokesman  stated 
that  the  physicians  did  not  interact  with  the  system.  )  There 
was  no  statistically  significant  difference  between  the  work 
groups.  However,  the  ancillary  group  was  more  satisfied  than 
the  administrative  group.  One  problem  with  this  assessment  is 
that  the  ancillary  group's  surveyed  population  only  composed 
approximately  three  and  one  half  percent  of  the  total 
respondent  population  as  opposed  to  the  administrative  group's 
overwhelming  96.5  percent.  Therefore,  the  ancillary  group's 
results  may  not  be  an  accurate  representation  of  the  actual 
group  population  as  a  whole. 

Both  administrative  and  ancillary  personnel  had  very 
low  regards  for  the  timeliness  of  outside  services  and 
modifications  to  the  system,  as  well  as  the  amount  of  training 
they  received.  However,  their  collective  satisfaction  with 
the  local  IMD  staff  was  considerably  higher.  The  users 
appeared  to  recognize  that  the  local  staff  was  both  willing 
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and  capable  of  performing  services  which  were  not  governed  by 
higher  authority. 

3.  Significant   Individual   Question   Differences   and 
Grouped  factors 

There  was  no  statistically  significant  difference 
between  the  work  groups  for  any  of  the  responses  to  the 
individual  questions  or  grouped  factors.  The  ancillary  group 
generally  appeared  to  be  more  satisfied  in  the  individual 
question  areas  as  well  as  the  four  grouped  factors.  However, 
as  suggested  earlier,  this  may  be  attributed  to  the  ancillary 
group's  size. 

4.  Satisfaction  Versus  Time  of  System  Use 

The  ancillary  group  only  had  users  with  six  to  11 
months  of  system  use,  however,  their  level  of  satisfaction  was 
relatively  high.  The  administrative  group's  level  of 
satisfaction  actually  increased  as  the  respondents'  time  of 
system  use  increased.  This  trend  is  consistent  with  that 
which  was  predicted  in  chapter  VII.  Many  respondents  equated 
experience  with  the  system  to  being  able  to  efficiently  and 
effectively  manipulate  the  system  and  further  translated  that 
to  "job  satisfaction". 

D.   PENSACOLA  AND  FORT  ORD  OVERALL  SATISFACTION 

A  comparison  of  the  satisfaction  levels  of  the  respondents 
from  the  both  systems  will  further  aid  in  future  evaluations. 
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1 .  Combined  Groups 

Combining  both  Pensacola  and  Fort  Ord  ' s  work  groups 
produces  an  average  satisfaction  index  of  9.64.  The  results 
indicated  that  there  was  a  low  level  of  satisfaction  among  all 
AQCESS  respondents. 

2.  Comparison  of  Work  Groups 

The  work  groups  identified  in  this  study,  included 
both  Pensacola' s  and  Fort  Ord's  administrative  and  ancillary 
groups.  The  administrative  group  displayed  the  highest  level 
of  satisfaction,  followed  by  the  ancillary  group.  It  has  been 
established  that  the  administrative  group's  use  of  the 
information  is  less  critical  than  that  of  the  ancillary  group. 
As  a  result,  they  are  less  frustrated  with  slow  response  and 
down  time  of  the  system. 

3.  Significant   Individual   Question   Differences   and 
Grouped  factors 

The  combined  work  groups  at  Pensacola  showed  a 
statistically  significant  higher  difference  than  Fort  Ord  for 
question  2.  It  would  appear  that  neither  site  is  at  all 
satisfied  with  the  way  in  which  outside  contractors  or  DMSSC 
process  requests  for  changes  to  the  system.  Some  users 
expressed  dissatisfaction  over  not  having  their  requests 
honored  in  a  timely  manner.  Others  understood  and  agreed  with 
the  process  involved  in  implementing  changes.  After  all,  if 
all  requests  were  acknowledged  and  implemented,  there  would 
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probably  be  a  number  of  unique  AQCESS  systems  throughout  DoD. 
There  was  no  statistically  significant  differences  regarding 
the  grouped  factors. 

E.   FORT  ORD,  PENSACOLA  AND  CHARLESTON  OVERALL  SATISFACTION 

Current  plans  for  AQCESS  include  many  new  initiatives. 
AQCESS  6.0  was  tested  and  accepted  by  representatives  from  the 
three  services  in  March,  1991.  Expanded  functions  have  been 
incorporated  into  AQCESS  7.0  which  include  scrolling  help 
windows  with  "point  and  shoot",  data  entry  from  help  tables 
and  Ad  Hoc  Reporting.  Beta  testing  is  expected  to  begin  the 
during  the  first  quarter  of  fiscal  year  1992.  DoD  funding  has 
been  established  to  support  hardware  upgrades  at  selected 
sites  in  fiscal  years  1992  and  1993.  Despite  all  these 
upgrades  to  the  AQCESS  system,  scheduling  in  September  and 
December  of  1992  the  AQCESS  systems  at  Pensacola  and  Fort  Ord 
will  be  replaced  by  the  CHCS  system.   (NMDSC,  1991) 

A  comparison  of  the  satisfaction  levels  of  the  respondents 
from  the  combined  AQCESS  systems  and  the  CHCS  system  will  aid 
in  future  evaluations. 

1 .   Combined  Groups 

The  survey  average  satisfaction  index  is  9.64. 
Charleston  is  more  satisfied  with  their  system  than  Fort  Ord 
or  Pensacola.  Pensacola  shows  a  less  than  average  level  of 
satisfaction.  This  may  be  attributed  to  the  fact  that  the 
Pensacola   physicians   displayed   a   very   low   level   of 
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satisfaction  for  the  AQCESS  system.   The  differences  in  the 
levels  of  satisfaction  were  not  statistically  significant. 

2.   Significant   Individual   Question   Differences   and 

Grouped  factors 

This  section  will  only  discuss  those  questions  that 
were  statistically  significant. 

a.  Processing  of  Requests  for  Changes  to  the  Existing 
System 

All  sites  responded  negatively.  Charleston  is  less 
satisfied  than  the  other  sites  and  happens  to  be  a  beta  test 
site  for  the  CHCS  system.  Software  support  for  this  site  is 
furnished  by  a  civilian  contractor.  The  CHCS  system  is  a 
relatively  new  system  under  development,  whereas  the  AQCESS 
system  has  been  operational  since  1984.  The  developers  and 
service  personnel  of  the  AQCESS  system  have  had  the 
opportunity  to  identify  and  correct  many  software  problems. 
The  AQCESS  systems'  MID/IMD  personnel  are  less  concerned  with 
system  implementation  and  more  concerned  with  system 
maintenance,  so  there  are  fewer  requests  for  changes. 
Therefore,  the  users  of  the  AQCESS  system  may  be  less 
frustrated. 

b.  Degree  of  Training  Provided  to  Users 
Charleston  was  more  satisfied  with  their  level  of 

training  than  the  other  sites.   Since  Charleston  was  a  test 
site  and  was  in  the  implementation  phase,  the  users'  formal 
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training  was  more  extensive  than  the  other  sites.  Therefore, 
Charleston  was  expected  to  show  a  higher  level  of 
satisfaction. 

c.  User's  Feeling  of  Participation 

Charleston  was  more  satisfied  with  their 
involvement  with  the  CHCS  system  than  the  users  of  the  AQCESS 
system.  This  can  be  expected,  especially  when  comparing  the 
physicians  of  both  sites.  The  CHCS  physicians  use  the  system 
for  direct  patient  care.  In  fact,  the  CHCS  physicains  have  a 
terminal  located  in  their  office.  Whereas,  the  AQCESS 
physicians'  only  legitimate  way  of  interfacing  with  the  system 
is  by  reviewing  statistics  of  their  performance  compiled  by 
the  Quality  Assurance  personnel. 

d.  Attitude  of  the  Information  Management  Staff 
Charleston  was  more  satisfied  with  the  Information 

Management  staff.  Since  they  are  a  test  site,  the  MID  staff 
provided  more  recent  and  extensive  training.  The 
communication  between  the  CHCS  users  and  their  staff  was 
expected  to  be  more  satisfying. 

e.  Accuracy  of  Output  Information 

The  accuracy  of  output  information  for  the  CHCS 
physicians,  in  many  instances,  is  directly  related  to  the 
information  that  they  input.  The  AQCESS  physicians  have 
little  or  no  control  over  the  input  information.   Therefore, 
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it  is  reasonable  to  assume  that  the  level  of  satisaction  for 
CHCS  users  will  be  higher. 

f.    Factor  A   (Local  MID/IMD  Staff  and  Services) 

Charleston  shows  a  higher  level  of  satisfaction. 
The  CHCS  users'  perception  of  the  local  MID  is  considered  to 
be  quite  satisfactory.  There  does  not  appear  to  be  any 
communication  problems  between  the  CHCS  users  and  the  MID 
staff.  The  AQCESS  users  appear  to  be  satisfied  with  the  level 
of  service  from  their  MID/IMD  staff,  but  to  a  lesser  degree. 
The  higher  level  of  satisfaction  for  the  CHCS  group  can  be 
attributed  to  the  increased  communication  required  between 
them  and  their  MID  staff  while  undergoing  testing  of  the 
system. 

F.   COMBINED  CHCS  AND  AQCESS  OVERALL  SATISFACTION 

The  work  groups  from  the  AQCESS  and  the  CHCS  systems  have 
been  combined  to  pinpoint  problems  of  a  particular  work  group. 
The  following  sections  will  analyze  the  responses  from  the 
combined  work  groups. 

1.   Overall  satisfaction 

The  overall  satisfaction  index  was  10.14.  This  places 
the  overall  satisfaction  index  in  the  lower  third  of  the 
scale.  This  score  should  be  used  as  a  baseline  measure  for 
follow-on  analysis  of  these  hospital  information  systems. 

The  ancillary  and  administrative  groups,  with 
comparable   satisfaction   indices,   appear   to   be   slightly 
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satisfied  with  their  information  systems.  The  physicians, 
with  a  near  neutral  satisfaction  index,  can  be  considered  to 
be  unsatisfied  with  the  system. 

2.   Significant  Individual  Questions 

The  following  section  will  only  address  those 
questions  with  statistically  significant  differences  among  the 
work  groups.  The  individual  questions  will  be  combined  into 
grouped  factors  and  discussed  in  the  next  section. 

a.  Processing    of    Requests    for    Changes    to    Existing 
Systems 

The  administrative  group  shows  a  higher  level  of 
satisfaction.  However  the  results  show  a  neutral  satisfaction 
index  for  the  administrative  group  and  negative  satisfaction 
indices  for  the  ancillary  group  and  physicians.  This  implies 
that  the  work  groups  are  dissatisfied  with  the  contracted 
services,  which  is  consistent  with  previous  analysis. 

b.  Degree  of  Training  Provided  to  Users 

The  physicians  consistently  demonstrated  a  lower 
level  of  satisfaction  for  training  in  all  the  previous 
comparisons.   Clearly,  this  is  an  area  for  further  study. 

c.  User's  Understanding  of  Systems 

The  low  level  of  satisfaction  is  evident  among  all 
work  groups  and,  in  particular,  the  physicians.  This  is 
consistent  with  previous  findings  that  suggests  users  with 
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better  computer  background  and  training  are  more  likely  to  be 
more  satisfied  with  the  system. 

d.  User's  Feeling  of  Participation 

As  previously  noted,  the  users  who  are  more 
involved  with  the  system  gain  a  better  understanding  of  the 
system.  As  a  result,  are  more  satisfied.  In  the  case  of  the 
AQCESS  physicians,  they  are  rarely  involved  with  the  system. 
Contrarily,  the  CHCS  physicians  with  terminals  in  their 
offices,  may  find  the  system  to  be  a  distraction  from  their 
daily  routine.  Thus,  the  physicians'  negative  satisfaction 
index  indicates  their  dissatisfaction  for  their  current  level 
of  participation. 

e.  Attitude  of  the   Information  Management   Staff 

For  this  item,  the  physicians  indicated 
satisfaction  with  the  system,  even  though  they  displayed  a 
lower  level  than  the  other  groups.  This  is  the  highest 
satisfaction  index  recorded  by  the  physicians.  The 
administrative  and  ancillary  groups  were  highly  satisfied  with 
the  attitude  of  their  MID/IMD  staffs. 

f.  Reliability  of  Output   Information 

The  physicians  are  not  satisfied  with  the 
reliability  of  output  information.  This  can  be  attributed  to 
their  roles  and  involvement  with  their  systems.  The 
administrative  and  ancillary  personnel  are  only  slightly 
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satisfied.   If  the  information  is  inconsistent,  the  users  will 
have  a  low  level  of  satisfaction. 

g.    Relevancy     of     Output      Information      (to      Intended 
Function 

The  physicians  perceived  the  output  information  as 
not  being  what  they  required.  The  administrative  and 
ancillary  groups  were  reasonably  satisfied. 

h.    Time  Required  for  New  Systems  Development 

Contractor  services  and  time  required  for  system 
changes  are  viewed  unfavorably  by  the  work  groups.  Responses 
to  requests  for  changes  to  information  systems  that  are 
controlled  by  a  central  management  (DMSSC)  are  more  likely  to 
be  slower  than  responses  for  an  information  system  that  has  a 
decentralized  management. 

i.    Completeness  of  the  Output  Information 

All  groups  show  a  positive  satisfaction  index. 
However,  consistent  with  previous  analysis,  the  physicians 
perceived  the  information  to  be  incomplete.  If  the 
information  is  perceived  to  be  inaccurate,  irrelevant  and 
inconsistent,  users  will  not  be  satisfied  with  the  output 
information. 

3.   Comparison  of  Grouped  Factors 

The  physicians  are  significantly  less  satisfied  for 
all  factors.  That  could  be,  for  the  most  part,  due  to 
combining  the  CHCS  physician  responses  with  those  of  the 


131 


AQCESS  physicians.  Although  the  sample  populations  for  the 
two  physician  groups  differ  by  only  one,  the  extremely 
negative  scores  reported  by  the  Pensacola  group  significantly 
influenced  the  overall  low  scores  of  the  combined  physician 
group.  Nevertheless,  the  physicians  tend  to  show  a  positive 
level  of  satisfaction  for  the  MID/IMD  staff  and  services  along 
with  the  administrative  and  ancillary  groups. 

The  analysis  of  the  responses  form  the  work  groups  for 
contractor  services  and  time  required  for  system  changes  are 
consistent  with  other  analysis.  The  administrative  and 
physician  groups  view  the  contractor  services  negatively, 
while  the  ancillary  group  is  neutral.  This  was  the  most 
frequent  cause  of  dissatisfaction  for  the  users. 

The  administrative  and  ancillary  groups  were  slightly 
satisfied  with  the  quality  of  output  delivered  by  the  system. 
The  physicians,  however,  were  dissatisfied. 

The  physicians  were  dissatisfied  with  knowledge  of  and 
involvement  with  the  system.  If  users  are  lacking  in 
training,  understanding,  and  experience  with  the  system,  they 
are  more  likely  to  be  less  satisfied. 
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XII.   CONCLUSIONS  AND  RECOMMENDATIONS 


A.   MEASURING  USER  INFORMATION  SATISFACTION 

Today,  there  is  a  greater  need  to  understand  the  user's 
point  of  view.  As  the  availability  of  hardware  and  software 
continues  to  grow,  it  becomes  more  difficult  to  remain  current 
with  the  pace  of  technological  changes.  Because  of  personal 
computing,  users  are  far  more  knowledgeable.  They  are  less 
tolerant  of  complex  time-consuming  processes  and  procedures, 
delays  due  to  backlogs  and  shifting  priorities  and  inadequate 
procedures  and  information  on  available  resources.  (Conrad  and 
Mignen,  1990,  Cash  et  al,  1988,  Mensching  and  Adams,  1991) 
Conrad  and  Mignen  (1988)  argue  that  the  focus  should  not  be  on 
the  cure  for  user  dissatisfaction,  but  to  develop  the  means  to 
identify  problems  before  they  occur.  This  is  almost 
impossible  without  some  means  of  gauging  users'  perceptions. 
The  user  information  satisfaction  survey  questionnaire 
provides  the  means  for  gauging  users'  perceptions.  The  survey 
questionnaire  is  easily  and  quickly  administered  and  provides 
a  standard  measure  for  comparisons  of  scores  across 
departments,  systems,  users,  organizations,  and  industries. 
This  survey  is  not  a  conclusive  evaluation  of  the  AQCESS  and 
CHCS  systems,  but  coupled  with  further  investigation,  can  be 
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a  powerful  tool  in  the  analysis  and  interpretation  of  the 
causes  of  user  dissatisfaction. 

B.   SATISFACTION  CHARACTERISTICS  OF  AQCESS 

Overall  satisfaction  is  a  generalization  of  all  the 
characteristics  that  effect  the  satisfaction  of  the  user. 
Along  with  overall  satisfaction,  the  study  specifically  looked 
at  the  four  factors  that  make  up  satisfaction. 

1.  Overall  Satisfaction 

The  survey  results  indicate  the  users  are  slightly 
satisfied  with  AQCESS  at  Silas  B.  Hays  Army  Hospital,  Fort 
Ord,  and  by  Naval  Hospital,  Pensacola.  The  physicians' 
responses  were  consistently  scored  low  across  CHCS  and  AQCESS. 
The  lack  of  Fort  Ord  physician  responses  to  the  survey  may 
have  caused  a  slightly  inflated  overall  satisfaction  score  for 
them.  The  overall  satisfaction  scores  across  organizations 
and  systems  did  not  differ  significantly. 

2.  Areas  of  Satisfaction 

All  three  work  groups  (to  a  lessor  extent  the 
physicians)  rated  the  local  MID/IMD  staff  and  services  as 
being  satisfactory.  However,  the  CHCS  users'  satisfaction 
index  is  twice  that  of  the  AQCESS  users'.  This  suggests  that 
the  local  AQCESS  MID/IMD  personnel  are  not  communicating  and 
interacting  with  their  users  as  effectively  as  the  CHCS  MID 
personnel  are  with  their  users.  CHCS ' s  success  in  this  area 
can  be  attributed  to  the  emphasis  on  training  the  users  during 
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the  implementation  phase  of  the  CHCS  system.  The  AQCESS 
systems'  MID/IMD  personnel,  far  beyond  the  implementation 
phase,  have  no  need  to  intensify  the  training  of  their  users. 
However,  they  do  maintain  a  solid  steady  training  program  that 
includes  both  formal  and  on-the-job  training. 

In  many  areas,  the  AQCESS  systems'  administrative 
groups  displayed  higher  satisfaction  than  any  other  group. 
The  ancillary  groups,  whose  interaction  with  the  system  is 
more  essential  ( regarding  direct  patient  care ) ,  were  more 
dissatisfied  because  of  the  system's  response  time. 
3.   Areas  of  Dissatisfaction 

Armed  with  a  set  of  problem  areas,  the  MTD/IMD  staff 
and  functional  managers  of  the  AQCESS  systems  can  explore  the 
underlining  causes  through:  (1)  interviews  and  (2)  by 
examining  the  development  and  operating  procedures  of 
particular  user  groups  within  their  systems.  Investigation 
into  areas  of  dissatisfaction  can  provide  context,  history  and 
insight  for  possible  corrective  measures. 

Since  the  physicians'  satisfaction  indices  were  low, 
across  the  organizations,  it  seems  to  be  a  characteristic  of 
physicians  to  display  a  low  level  of  satisfaction  for  hospital 
information  systems.  The  limited  involvement  of  the  Pensacola 
physicians  was  a  major  factor  for  their  overall  low  level  of 
satisfaction. 
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The  respondents  were  most  dissatified  with  the  amount 
of  time  it  took  for  new  system  development  and  changes  to 
occur.  AQCESS  is  a  centrally  managed  information  system, 
therefore  it  requires  a  relatively  long  time  to  process 
requests  for  changes. 

4.   Satisfaction  with  Time  of  System  Use 

Regression  analysis  revealed  no  correlation  between 
satisfaction  and  time  of  system  use.  The  Fort  Ord 
administrative  group  displayed  increased  satisfaction  with 
increased  time  of  use.  All  Pensacola  groups,  with  greater 
than  11  months  of  experience,  were  less  satisfied. 

C .   RECOMMENDATIONS 

The  short- form  questionnaire  developed  by  Baroudi  and 
Orlikowski  (1988)  is  an  effective  means  to  measure  user 
information  satisfaction.  This  survey  instrument  is  an 
appropriate  instrument  to  document  user  satisfaction  within 
Medical  Treatment  Facilities  as  well  as  documenting  changes  in 
user  satisfaction  at  Naval  Hospitals,  Pensacola  and  Silas  B. 
Hays  (Army)  Hospital  at  a  later  time. 

This  type  of  survey,  to  prevent  biased  results,  is  best 
conducted  by  a  researcher  who  is  not  experienced  with  the 
system.  Thus,  the  interpretation  of  the  results  are  more 
likely  to  reflect  impartiality.  It  is  recommended  that  the 
researcher  arranges  with  a  member  of  the  organization  to  serve 
as  a  point  of  contact.   The  contact  distributes  and  collects 
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the  questionnaires  for  the  researcher.  It  should  be 
emphasized  prior  to  the  survey  that  the  questionnaire  elicits 
responses  reflective  of  the  present  conditions  and  not  the 
past  conditions  and  experiences  with  information  systems  or 
MID/IMD  staffs.  In  employing  the  user  satisfaction 
questionnaire,  it  was  found  that  some  individuals  were 
uncertain  as  to  the  exact  meaning  of  certain  questions.  If 
the  lack  of  clarity  is  likely  to  be  a  problem,  it  is 
recommended  that  full  explanations  of  the  scales  be  included. 
Additionally,  direct  contact  with  individuals  for  interview  by 
the  researcher  is  beneficial  for  adding  context  to  history. 
Additionally,  the  following  is  recommended  prior  to  conducting 
the  interview: 

1 .  Structure  the  interviews  around  known  problem  areas  to 
avoid  "orienting"  time  spent  in  searching  for  real 
issues. 

2.  Avoid  focusing  on  the  highly  specific,  idiosyncratic 
complaints  of  individual  users  that  are  not  of  general 
concern. 

3 .  Reduce  the  number  of  interviews  needed  to  obtain  deeper 
undertsanding  of  the  problem  areas. 

To  obtain  a  deeper  understanding  of  the  issues  by  adding 

context  to  history,  it  is  also  recommended  that  the  following 

items  be  further  investigated: 

1.  The  administrative  group's  overall  higher  satisfaction 
over  the  other  work  groups. 

2.  The  physicians'  lowered  perception  of  satisfaction  in 
almost  all  areas. 

3.  The  reasons  for  the  respondents  having  negative 
perceptions  of  satisfaction  with  contractor's  services. 
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The  user  satisfaction  questionnaire  should  be  used  as  a 
standard  measuring  tool.  This  tool  will  allow  both  the 
researcher  and  the  practitioner  to  utilize  a  readily  available 
instrument,  thus,  avoiding  the  process  of  developing  a  new 
measure  each  time  an  assessment  of  user  satisfaction  is 
required.  A  follow  on  study  should  be  conducted  at  Silas  B, 
Hays  Hospital,  Fort  Ord  and  Naval  Hospital,  Pensacola,  using 
the  results  of  this  study  as  a  baseline  comparison.  Finally, 
the  results  of  this  study  should  be  used  for  measuring  user 
satisfaction  at  other  DoD  hospital  sites. 
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APPENDIX  A 

Part  A:  General  Information 

1.   Hospital  Division/Department:  (Check  one) 

General  Administration 

Nursing  Administration 

Dietary 

Emergency 

Laboratory 

Medical  Clinic 

Inpatient  Nursing 

Pharmacy 

Radiology 

Other  (Specify): 


2.   Job  Description:  (Check  one) 

Administration 

Medic 

Technician 

Nurse 

Pharmacist 

Physician 

Physician  Assistant 

Other  (Specify): 


3.   Highest  Level  of  Education:  (Check  one) 

High  School  Graduate 

Some  College 

Associate's  Degree 

Bachelor's  Degree 

Some  Graduate  Work 

Master's  Degree 

Doctoral  Degree 

Medical  Degree 

Other  (Specify): 


4 .   Age : 


5.  Gender:  Male  Female 

6.  Length  of  time  (in  months)  you  have  used  AQCESS: 


7.  Have  you  used  other  computer  systems  before  ?   Yes  No 

8.  If  your  answer  was  Yes  to  question  7,  was  it  a  health  care 
information  system  ?   Yes  No 


139 


Part  B:    User  Satisfaction  Questionnaire 

This  section  conveys  your  personal  feelings  concerning  the  use 
of  the  Automated  Quality  of  Care  Support  System  at  Naval  Hospital, 
Pensacola  /  Silas  B.  Hays  Hospital  (Fort  Ord ) .   Please  do  not 
attempt  to  analyze  the  questions.   Remember,  there  are  no  right  01 
wrong  answers. 

Please  follow  these  instructions: 

1 .   Check  each  scale  in  the  position  that  describes  your 

evaluation  or  answer.   Example:  Inventor  of  the  traffic  light 

genius  :_JL : : : 2_: : : 3_:  incompetent 

You  would  check  an  "X"  in  the  left-hand  side  of  the  scale 
(marked  as  "1"  in  the  above  example)  if  you  thought  the 
inventor  was  a  genius.   Similarly,  you  would  mark  the 
mid-point  of  the  scale  ("2"  in  the  above  example)  if  you 
thought  the  inventor  was  neither  a  total  genius  nor  totally 
incompetent  and  the  right-hand  side  (point  "3"  in  the  above 
scale)  if  you  thought  the  inventor  was  incompetent.   The  other 
positions  can  be  used  to  indicate  varying  degrees  between 
"1"  and  "2"  or  "2"  and  "3". 


2.  Check  in  the  space,  not  between  spaces  (be  sure  to 
check  only  one  position  per  scale). 

(Correct  way  -*  :  x  :    Incorrect  way  -*  : x :  ) 

3.  Check  both  of  the  scales  after  each  question. 

4.  Work  rapidly,  do  not  omit  any  questions  and  rely  on  your 
first  impressions. 


ANSWERS  BASED  ON  YOUR  OWN  OPINIONS 

Relationship  with  the  Management  Information  Department  (MID) 
Information  Management  Division  (IMD)  staff 

dissonant  : : : : : : : :  harmonious 

bad  : : : : : : : :  good 

!.   Processing  of  requests  for  changes  to  existing  systems 

fast  : : : : : : : :  slow 

untimely  : : : : : : : :  timely 

I.   Degree  of  training  provided  to  users 

complete  : : : : : : : :  incomplete 

low  : : : : : :_  _: :  high 

i.   User's  understanding  of  systems 

insufficient  : : : : : : : :  sufficient 

complete  : : : : : : : :  incomplete 
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5.  User's  feeling  of  participation 

positive  : : : : : : : :  negative 

insufficient  : : : : : : : :  sufficient 

6.  Attitude  of  the  Management  Information  Department  /  Information 
Management  Division  staff 

cooperative  : : : : : : : :  belligerent 

negative  : : : : : : : :  positive 

7.  Reliability  of  output  information 

high  : : : : : : : :  low 

superior  : : : : : : : :  inferior 

8.  Relevancy  of  output  information  (to  intended  function) 

useful  : : : : : : : :  useless 

relevant  : : : : : _: : :  irrelevant 

9.  Accuracy  of  output  information 

inaccurate  : : : : : : : :  accurate 

low  : : : : : : : :  high 

10.  Precision  of  output  information 

low  : : : : : : : :  high 

definite  ::::::::  uncertain 


11.  Communication  with  the  Management  Information  Department  / 
Information  Management  Division  staff 

dissonant  : : : : : : : :  harmonious 

destructive  : : : : : : : :  productive 

12.  Time  required  for  new  systems  development 

unreasonable  : : : : : : : :  reasonable 

acceptable  : : : : : : : :  unacceptable 

13.  Completeness  of  the  output  information 

sufficient  : : : : : : : :  insufficient 

adequate  : : : : : : : :  inadequate 


Thank  you  again  for  your  cooperation 
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APPENDIX  B 

Automated  Quality  of  Care  Evaluation 
Support  System  Survey 


This  survey  is  part  of  a  study  of  the  Automated  Quality  of  Car 
Evaluation  Support  System  (AQCESS)  and  its  effectiveness  in 
military  hospital  settings.   The  purpose  of  the  study  is  to  obtain 
information  regarding  your  perceptions  or  views  of  how  well  AQCESS 
functions  in  your  specific  area  of  use. 

Just  a  few  minutes  are  required  to  fill  out  general  informatio 
along  with  a  13  question  survey.   Your  identity  will  remain 
confidential.   The  data  gathered  through  the  survey  will  provide 
valuable  insight  into  the  system's  strengths  and/or  weaknesses  as 
well  as  assist  in  future  research,  development  and  training. 

The  success  of  this  survey  depends  on  my  receiving  as  many 
completed  surveys  as  possible  from  users  in  all  hospital  areas. 
Your  participation  lends  an  important  contribution  towards  this 
end.   There  are  no  right  or  wrong  answers,  only  your  candid 
response  to  each  question.   Please  do  not  omit  any  of  the  question 
in  either  section  of  the  survey. 

Please  return  your  completed  survey  to  LCDR  Neeley  in  the 
Management  Information  Department  /  Mr.  G.  Scott  (HSXT-IMD)  in  the 
envelope  provided.   Thank  you  for  your  cooperation. 


James  R.  Booth,  LCDR,  USN  /  John  L.  Bryant,  Jr.,  LT,  US 

Naval  Postgraduate  School 
SMC  1073  /  SMC  2604 

Monterey,  CA   93940-4831 
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APPENDIX  C-l 


NAVAL  HOSPITAL  PENSACOLA 

COMBINED  RESULTS  FOR  PHYSICIANS  VS  ANCILLARY  VS 
ADMINISTRATIVE 


TOTAL  SATISFACTION 

PHYSICIAN 

ADMIN 

ANCILLARY 


AVERAGE  INDEX 
-0.3899  -4.15909 
1.42659  12.83929 
0.72624  11.49265 


GROUP  FACTORS 


A 
(MID) 


A 
(CON) 


B 


PHYSICIAN  SAT 
ANCILLARY  SAT 
ADMIN  SAT 
PHYSICIAN  STDEV 
ANCILLARY  STDEV 
ADMIN  STDEV 


0.64394  -0.76136  -0.4955  -1.01136 

1.41176  0.003676  0.70882  0.376838 

1.7123  0.309524  1.00357  0.59375 

1.43618  1.538883  1.53888  1.595408 

1.52979  1.694494  1.73813  1.816868 

1.44426   1.76929  1.69909  1.828224 


INDIVIDUAL  QUESTION  RESPONSES 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


12  3         4        5 

0.63636  -1.04545  -1  -0.75  -1.25 

1.5  -0.09504  0.13971  0.69117  0.82353 

1.72024  0.309524  0.19048  0.95833  0.91667 


STAN  DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 


12        3         4        5 

1.58245  1.445283  1.43019  1.79804  1.63936 

1.53872  1.800855  1.85968  1.75530  1.67104 

1.40121  1.883362  1.81578  1.80394  1.67053 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


6         7        8         9       10 

0.5  -0.61364  -0.6591  -0.4772  -0.3182 

1.39706  0.941176  0.91176  0.61029  0.35294 

1.88095  0.952381  1.30357  0.97023  0.68452 


STAN  DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 


6         7        8         9       10 

1.58831  1.555283  1.46075  1.42204  1.60642 

1.57312  1.670526  1.67363  1.73702  1.68265 

1.41762  1.76544  1.52268  1.72318  1.70133 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


11        12  13 

0.79545  -0.47727  -0.4091 

1.33824  0.154412  0.72794 

1.53571  0.213178  1.10714 
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STAN  DEV  11         12       13 

PHYSICIAN  1.057  1.097283  1.61412 

ANCILLARY  1.4714  1.566556  1.85293 

ADMIN  1.49161  1.647337  1.85293 
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APPENDIX  C-2 


NAVAL  HOSPITAL  PENSACOLA 


PHYSICIANS  VS  THE  REST  OF  THE  SURVEY  GROUP 


GENERAL 

PHYSICIAN 
STDEV 

NON-PHYSICIAN 
STDEV 


AVERAGE 
-0.3899 
1.62264 
0.8707 
1.75216 


INDEX 
-4.1591 
9.4067 
13.7796 
20.3155 


GROUP  FACTORS 

PHYSICIAN  GROUP 
NON-PHYSICIAN 
PHYSICIAN  STDEV 


A 

(MID) 

0.64394 

1.57785 

1.43618 


A 
(CON) 
-0.76140 
0.17270 
1.31421 


B  C 

-0.49550  -1.0114 
0.87171   0.49671 


NON-PHYSICIAN  STDEV   1.49064   1.74287 


0.28359 
1.72291 


0.31756 
1.82634 


QUESTION  1         2 

PHYSICIAN  SAT  0.63636  -1.04550 

PHYSICIAN  STDEV  1.58245   1.44528 

NON-PHYSICIAN  SAT  1.62171   0.10526 

NON-PHYSICIAN  STD  1.46841   1.86081 


3         4         5 

■1.0000  -0.7500  -1.2500 

1.4301   1.7980  1.63936 

0.1677   0.8388  0.87500 

1.8357   1.7872  1.67140 


QUESTION  6         7 

PHYSICIAN  SAT  0.50000  -1.0455 

PHYSICIAN  STDEV  1.58831   1.55528 

NON-PHYSICIAN  SAT  1.66447   0.94737 

NON-PHYSICIAN  STD  1.50850   1.72363 


8  9  10 
1.0000  -0.6136  -0.3182 
1.4607  1.4220  1.49033 
1.1282  0.8092  0.53618 
1.6038   1.7386  1.70101 


QUESTION  11        12        13 

PHYSICIAN  SAT  0.79545  -0.47730  -0.4091 

PHYSICIAN  STDEV  1.05700   1.09728   1.61412 

NON-PHYSICIAN  SAT  1.44737   0.24013   0.8914 

NON-PHYSICIAN  STD  1.48585   1.61354   f. 78611 
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APPENDIX  C-3 


PHYSICIANS  vs  THE  REST  OF  THE  SURVEY  GROUP 


CALCULATED  T-VALUE 


1 

2.91074* 

2 

2.78081* 

3 

2.86054* 

4 

3.89161* 

5 

5.73435* 

6 

3.34561* 

7 

5.13898* 

8 

5.89105* 

9 

3.60145* 

10 

2.24833* 

11 

2.00319* 

12 

2.02872* 

13 

3.22161* 

FACTOR 

A(MID) 

2.77688* 

A(CON) 

2.41541* 

B 

3.71987* 

C 

3.91692* 

OVERALL  4.07910* 

*  SIGNIFICANT  AT  ALPHA  = 


.05 
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APPENDIX  C-4 


NAVAL  HOSPITAL  PENSACOLA 

ANCILLARY  VS  THE  REST  OF  THE  SURVEY  GROUP 


GENERAL 
ANCILLARY 
NON- ANCILLARY 
ANCILLARY  STDEV 
NON- ANCILLARY  STDEV 

GROUP  FACTORS 


ANCILLARY 
NON- ANCILLARY 
ANCILLARY  STDEV 
NON-ANCILLARY  STDEV 

QUESTIONS 

ANCILLARY 
NON- ANCILLARY 
ANCILLARY  STDEV 
NON-ANCILLARY  STDEV 

QUESTIONS 

ANCILLARY 
NON- ANCILLARY 
ANCILLARY  STDEV 
NON- ANCILLARY  STDEV 


AVERAGE  INDEX 
0.72624  11.49265 
0.70174  9.311321 
1.75436  21.45832 
1.80618  14.53046 


A 
(MID) 


A 
(CON) 


B 


1.41176  0.003676  0.70882  0.376838 

1.49057  0.087264  0.69245  0.175393 

1.52979  1.694494  1.73813  1.788388 

1.50625  1.740047  1.76784  1.897845 


1.5  -0.14706  0.13971  0.69117  0.82353 

1.49528  0.028302  -0.0566  0.60377  0.46698 

1.53872  1.800855  1.85968  1.75530  1.67104 

1.50627  1.883181  1.80842  1.93126  1.88185 


8 


10 


39706  0.941176  0.91176  0.61029  0.82353 
59434  0.627358  0.89623  0.66981  0.47642 
57312  1.670526  1.67363  1.73702  1.68265 


1.55877  1.837196  1.70697  1.76561  1.73053 


QUESTIONS 

ANCILLARY 
NON -ANCILLARY 
ANCILLARY  STDEV 
NON-ANCILLARY  STDEV 


11       12        13 

1.33471  0.154412  0.72794 

1.38208  0.146226  0.79245 

1.4714  1.566556  1.85293 

1.444  1.581835  1.79998 
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APPENDIX  C-5 

ANCILLARY  vs  THE  REST  OF  THE  SURVEY  GROUP 
CALCULATED  T-VALUE 


1 

0.00000 

2 

0.62229 

3 

-0.69054 

4 

-0.30014 

5 

-1.25072 

6 

0.78195 

7 

-1.12362 

8 

0.22088 

9 

0.21962 

10 

-1.27920 

11 

0.22167 

12 

0.23056 

13 

0.21223 

FACTOR 

A(MID)  0.34149 
A(CON)  0.31046 
B  -0.03660 
C      -0.69144 

OVERALL  -1.50880 


*  SIGNIFICANT  AT  ALPHA  =  .05 
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APPENDIX  C-6 


NAVAL  HOSPITAL  PENSACOLA 


ADMINISTRATIVE  VS  THE  REST  OF  THE  SURVEY  GROUP 


GENERAL 
ADMIN  SAT 
NON-ADMIN  SAT 
ADMIN  STDEV 
NON-ADMIN  STDEV 

GROUP  FACTORS 


AVERAGE 

INDEX 

0.98764 

12.83929 

0.45342 

7.666667 

1.74163 

13.53834 

1.78861 

20.36609 

A 

A 

(MID) 

(CON) 

B 


ADMIN 
NON-ADMIN 
ADMIN  STDEV 
NON-ADMIN  STDEV 


1.7123  0.309524  0.98148  0.59375 

1.22407  -0.18333  0.41444  0.0375 

1.44426  1.76929  1.69909  1.828224 

1.54314  1.643083  1.769  1.863385 


QUESTIONS 


ADMIN  SAT 
NON-ADMIN  SAT 
ADMIN  STDEV 
NON-ADMIN  STDEV 


1.72024  0.309524  0.19048  0.95833  0.91667 

1.28889  -0.36667  -0.1389  0.33888  0.31667 

1.40121  1.883362  1.81578  1.80394  1.67053 

1.59335  1.763519  1.8311  1.87131  1.88702 


QUESTIONS 


8 


10 


ADMIN  SAT 
NON-ADMIN  SAT 
ADMIN  STDEV 
NON-ADMIN  STDEV 


1.88095  0.952381 
1.17778  0.561111  0.52778 
1.41762   1.76544  1.52268 


30357  0.97023  0.68452 
0.34444  0.18889 
1.72318 


1.62329  1.773772  1.75888  1.72987 


70133 
68914 


QUESTIONS 

ADMIN  SAT 
NON-ADMIN  SAT 
ADMIN  STDEV 
NON-ADMIN  STDEV 


11 

12 

13 

1, 

.53571 

0, 

.309524 

1 

.10714 

1, 

.20556 

0, 

.096939 

0.45 

1, 

.49161 

1. 

.647337 

1 

.71143 

1, 

.40118 

1. 

.490712 

1 

.86272 
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APPENDIX  C-7 


ADMINISTRATIVE  vs  THE  REST  OF  THE  SURVEY  GROUP 


CALCULATED  T-VALUE 


1 

-1.88791 

2 

-2.46426 

* 

3 

-1.19177 

4 

-2.22523 

* 

5 

-2.21289 

* 

6 

-3.02213 

* 

7 

-1.45634 

8 

-3.07875 

* 

9 

-2.40038 

* 

10 

-1.90959 

11 

-1.50105 

12 

-0.92410 

13 

-2.43150 

* 

FACTOR 

A(MID) 

-2.16394 

* 

A(CON) 

-1.89548 

* 

B 

-2.16356 

* 

C 

-0.07198 

* 

OVERALL 

■1.45925 

*  SIGNIFICANT  AT  ALPHA  =  .05 
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APPENDIX  C-8 


NAVAL  HOSPITAL  PENSACOLA 
ANCILLARY  VS  ADMINISTRATIVE  GROUP 


GENERAL 
ANCILLARY  SAT 
ADMIN  SAT 
ANCILLARY  STDEV 
ADMIN  STDEV 


AVERAGE     INDEX 
0.72624     13.73530 
0.98764     13.81550 
1.75436     25.13573 
1.74163     15.34150 


GROUP  FACTORS 

ANCILLARY  SAT 
ADMIN  SAT 
ANCILLARY  STDEV 
ADMIN  STDEV 


A         A         B        ( 
(MID)     (CON) 
1.41176   0.003676  0.70882  0.37684 
1.71230   0.30952   0.98148  0.59375 
1.52979   1.694494  1.73813  1.78839 
1.44426   1.76929   1.69909  1.82822 


QUESTIONS 
ANCILLARY  SAT 
ADMIN  SAT 
ANCILLARY  STDEV 
ADMIN  STDEV 


1          2  3        4        5 

1.50000  -0.14710  0.13971  0.69118  0.82353 

1.72024  0.30952  0.19048  0.95833  0.91667 

1.53872  1.80086  1.85968  1.75531  1.67104 

1.40121  1.88336  1.81578  1.80394  1.67053 


QUESTIONS 
ANCILLARY  SAT 
ADMIN  SAT 
ANCILLARY  STDEV 
ADMIN  STDEV 

QUESTIONS 
ANCILLARY  SAT 
ADMIN  SAT 
ANCILLARY  STDEV 
ADMIN  STDEV 


1.39706 
1.88095 
1.57312 
1.41762 

11 
1.33471 
1.53571 
1.47140 
1.49161 


7 
0.94118 
0.95238 
1.67053 
1.76544 

12 
0.15441 
0.30952 
1.56656 
1.64734 


8 
0.91176 
1.30357 
1.67363 
1.52268 

13 
0.72794 
1.10714 
1.85293 
1.71143 


0.61029 
0.97024 
1.73702 
1.72318 


10 
0.82353 
0.68452 
1.68265 
1.70133 
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APPENDIX  C-9 


ANCILLARY  vs  ADMINISTRATIVE  GROUP 


CALCULATED  T-VALUE 


1 

0.84816 

2 

1.51933 

3 

1.66100 

4 

0.93099 

5 

0.36708 

6 

2.05345 

7 

0.03501 

8 

1.38279 

9 

1.27973 

10 

-0.50710 

11 

0.81736 

12 

0.56945 

13 

1.31714 

FACTOR 

A(MID) 

0.03318 

A(CON) 

1.08134 

B 

0.97679 

C 

0.68113 

OVERALL  0.030183 

*  SIGNIFICANT  AT  ALPHA  = 


.05 
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APPENDIX  C-10 

NAVAL  HOSPITAL  PENSACOLA 

SATISFACTION  BASED  ON  TIME  ON  THE  SYSTEM 

OVERALL 


TIME  OF 

#  PEOPLE 

USE 

AVG  SAT 

35 

1  TO  5 

11.94 

34 

6  TO  11 

12.66 

104 

>=  12 

7.58 

174 


ADMINISTRATIVE 


TIME  OF 

#  PEOPLE 

USE 

AVG  SAT 

19 

1  TO  5 

15.16 

17 

6  TO  11 

16.07 

48 

>=  12 

10.99 

84 

ANCILLARY 

TIME  OF 

#  PEOPLE 

USE 

AVG  SAT 

12 

1  TO  5 

9.96 

13 

6  TO  11 

13.71 

43 

>=  12 

9.45 

68 

PHYSICIANS 

TIME  OF 

#  PEOPLE 

USE 

AVG  SAT 

4 

1  TO  5 

10.07 

4 

6  TO  11 

3.54 

14 

>=  12 

6.36 

22 
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APPENDIX  D-l 

SILAS  B.  HAYS  ARMY  HOSPITAL 

COMBINED  RESULTS  FOR  ADMINISTRATIVE  AND  ANCILLARY 


TOTAL  SATISFACTION 
ADMIN 
STAN  DEV 
ANCILLARY 
STAN  DEV 

AVERAGE 
0.79860 
1.76524 
1.00000 
1.42774 

INDEX 

10.38182 

15.80275 

13.00000 

13.50000 

GROUP  FACTORS 

ADMIN  AVG 
ANCILLARY  AVG 
ADMIN  STDEV 
ANCILLARY  STDEV 

A 
(MID) 
1.27878 
1.54545 
1.74456 
1.83031 

0 
0 
1 
2 

A 
(CON) 
.05909 
.37500 
.76356 
.34187 

0. 
0. 
1. 

1. 

B 

94545 
90000 
67894 
17898 

0, 
0. 
1. 
1. 

c 

.34545 
.87500 
.77449 
.72753 

COMBINED  GROUP 

AVERAGE 
STAN  DEV 

FACTORS 

A 

(MID) 

1.28739 

1.73276 

0 

1 

A 
(CON) 
.07017 
.78796 

0. 
1. 

B 

94386 
66396 

0, 

1. 

C 

.36403 
.77554 

INDIVIDUAL  QUESTION  RESPONSE 


QUESTION 

ADMIN 

ANCILLARY 

1 
1.22727 
1.50000 

2 

-0.31818 
0 

3 
0.118182 
0.750000 

0. 

1. 

4 
.90909 
.50000 

5 
0.67272 
1.25000 

STAN  DEV 

ADMIN 

ANCILLARY 

1 
1.72499 
1.50000 

2 

1.73169 
3.00000 

3 
1.73588 
0.829156 

1. 
0. 

4 
.56405 
,50000 

5 
1.79448 
0.82915 

QUESTION 

ADMIN 

ANCILLARY 

6 
1.15454 
1.66666 

7 
0.95454 
1.25000 

8 
1.20909 
1.50000 

0. 
0 

9 
,91818 

10 
0.80000 
1.00000 

STAN  DEV 

ADMIN 

ANCILLARY 

6 
1.82501 
0.47140 

7 
1.691618 
0.829156 

8 
1.65746 
0.50000 

1. 
1. 

9 
,70630 
,41421 

10 
1.58286 
1.00000 

QUESTION 

ADMIN 

ANCILLARY 

11 
1.45454 
1.50000 

12 
0.43636 
0.75000 

13 
0.84545 
0.75000 

STAN  DEV 

ADMIN 

ANCILLARY 

11 
1.66589 
1.50000 

12 
1.71372 
1.29903 

13 
1.72250 
1.29903 
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APPENDIX  D-2 


ANCILLARY  vs  ADMINISTRATIVE 


CALCULATED  T-VALUE 


1 

-0.22114 

2 

-0.24072 

3 

-0.50956 

4 

-0.52982 

5 

-0.45345 

6 

-0.40038 

7 

-0.24837 

8 

-0.25320 

9 

0.75420 

10 

-0.17685 

11 

-0.04192 

12 

-0.25435 

13 

0.07299 

FACTOR 

A(MID) 

-0.16331 

A(CON) 

-0.24566 

B 

0.04106 

C 

-0.40956 

OVERALL  -0.23099 

*  SIGNIFICANT  AT  ALPHA  = 


05 
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APPENDIX  D-3 


SILAS  B.  HAYS  ARMY  HOSPITAL 


SATISFACTION  BASED  ON  TIME  OF  USE 


OVERALL 

TIME  OF 

#  PEOPLE 

USE 

12 

1  TO  5 

11 

6  TO  11 

34 

>=  12 

57 

AVG  SAT 

5.291667 
11.22727 
12.05882 


ADMINISTRATIVE 


#  PEOPLE 

TIME  OF 
USE 

AVG  SAT 

12 
9 

34 
55 

1  TO  5 
6  TO  11 
>=  12 

5.291667 
10.83333 
12.05882 

ANCILLARY 

#  PEOPLE 

TIME  OF 
USE 

AVG  SAT 

0 
2 
0 
2 

1  TO  5 
6  TO  11 
>=  12 

N/A 

13 

N/A 
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APPENDIX  E-l 


NAVAL  HOSPITAL  PENSACOLA  AND  SILAS  B.  HAYS  ARMY  HOSPITAL  FT 
COMBINED  RESULTS 


ORD 


OVERALL 

QUESTION 

INDEX 

AVERAGE 

0.7345987 

9.636364 

STAND  DEV 

1.7789492 

14.83246 

TOTAL  SATISFACTION 

QUESTION 

INDEX 

PHYSICIAN 

-0.38986 

-4.15909 

STAND  DEV 

1.6226442 

9.406703 

ADMIN 

0.912839 

11.86691 

STAND  DEV 

1.753449 

14.52653 

ANCILLARY 

0.7340659 

9.464789 

STAND  DEV 

1.7464702 

14.38562 

GROUP  FACTORS 

A 
(MID) 

A 
(CON) 

B 

c 

PHYSICIAN  SAT 

0.0606061 

-0.76136 

-0.49545 

-1.01136 

ANCILLARY  SAT 

1.4119048 

0.252381 

0. 

.714286 

0, 

.391071 

ADMIN  SAT 

1.5407674 

0.210432 

0. 

.980576 

0, 

.495504 

PHYSICIAN  STDEV 

1.4361806 

1.538883 

1. 

.538883 

1, 

.595408 

ANCILLARY  STDEV 

1.524202 

1.717497 

1, 

.724967 

1. 

.816273 

ADMIN  STDEV 

1.5842124 

1.771263 

1. 

.691382 

1.81123 

OVERALL  AVG 

1.4163059 

0.058442 

0. 

,759307 

0. 

.320346 

OVERALL  STDEV 

1.573773 

1.739173 

1. 

.717359 

1, 

,845252 

INDIVIDUAL  QUESTION  RESPONSES 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


1          2         3  4  5 

0.6363636  -1.04545     -1  -0.75  -1.25 

1.5  -0.14286  0.157143  0.714286  0.835714 

1.5251799  0.061151  0.161871  0.938849  0.820144 


STAN  DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 


1          2         3         4  5 

1.582445  1.445283  1.430194  1.798042  1.63936 

1.5376234  1.846121  1.841084  1.737345  1.654478 

1.5562866  1.850491  1.784945  1.713216  1.724774 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


6  7         8  9        10 

0.5  -0.61364  -0.65909  -0.47727  -0.31818 

1.3928571  0.95  0.928571  0.592857  0.371429 

1.5935252  0.953237  1.266187  0.94964  0.730216 


STAN  DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 


6  7         8         9        10 

1.5883096  1.555283  1.460747  1.422045  1.606418 

1.5569954  1.653244  1.654616  1.731624  1.670513 

1.6304966  1.736606  1.684424  1.716713  1.656431 
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QUESTION  11        12        13 

PHYSICIAN  0.7954545  -0.47727  -0.40909 

ANCILLARY  1.3428571  0.171429  0.728571 

ADMIN  1.5035971  0.359712  1.003597 

STAN  DEV  11        12        13 

PHYSICIAN  1.0570015  1.097283  1.614116 

ANCILLARY  1.4724684  1.562703  1.839421 

ADMIN  1.5634039   1.67507  1.839421 
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APPENDIX  E-2 


AQCESS  FT  ORD   vs  AQCESS  PENSACOLA 


OVERALL 

QUESTION 

INDEX 

PENSACOLA 

0.71132 

9. 

02586 

FT  ORD 

0.80567 

10. 

47368 

PENSACOLA  STDEV 

1.78615 

14. 

93465 

FT  ORD  STDEV 

1.75489 

15. 

73504 

GROUP  FACTORS 

A 
(MID) 

A 
(CON) 

B 

c 

PENSACOLA 

0.89719 

0 

.05452 

0.69885 

0 

.30582 

FT  ORD 

1.28363  • 

-0 

.03955 

0.94386 

0 

36403 

PENSACOLA  STDEV 

1.51598 

1 

.72164 

1.76039 

1 

.86665 

FT  ORD  STDEV 

1.13162 

1 

.77987 

1.65140 

1 

.77555 

INDIVIDUAL  QUESTION  RESPONSES 


QUESTION 
PENSACOLA 
FT  ORD 

STAN  DEV 
PENSACOLA 
FT  ORD 

QUESTION 
PENSACOLA 
FT  ORD 

STAN  DEV 
PENSACOLA 
FT  ORD 

QUESTION 
PENSACOLA 
FT  ORD 

STAN  DEV 
PENSACOLA 
FT  ORD 


1 
1 

.49713 
.23684 

-0.04011 
-0.30702 

0 
0 

.02012 
.14035 

0 
0 

.63793 
.92998 

0.60632 
0.69299 

1 

1 

1 
.51903 
.71834 

2 
1.85706 
1.79242 

1 
1 

3 
.83113 
.71616 

1 
1 

4 
.86497 
.54306 

5 
1.81078 
1.77273 

1 
1 

6 
.51724 
.15789 

7 
0.75000 
0.96491 

0 

1 

8 

.90210 
.21930 

0 
0 

9 

.64655 
.88596 

10 
0.42816 
0.80702 

1 
1 

6 
.56735 
.79951 

7 
1.78052 
1.66980 

1 
1 

8 

.69403 
.63170 

1 
1 

9 

.75474 
.70530 

10 
1.71304 
1.56652 

1 
1 

11 
.36494 
.45614 

12 
0.14943 
0.44444 

0 
0 

13 

.76724 
.84211 

1 
1 

11 

.45492 
66038 

12 
1.57589 
1.65018 

1 
1 

13 

.82113 
.70951 
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APPENDIX  E-3 


AQCESS  FT  ORD  vs  AQCESS  PENSACOLA 


CALCULATED  T-VALUE 


1 

-1.65958 

2 

0.92434 

3 

-0.43705 

4 

-1.06333 

5 

-0.32762 

6 

-1.34927 

7 

-0.82097 

8 

-1.17281 

9 

-0.86718 

10 

-0.47880 

11 

-0.26139 

12 

-1.17853 

13 

0.37824 

FACTOR 

A(MID) 

-1.84777 

A(CON) 

0.32150 

B 

-0.75588 

C 

-0.17464 

OVERALL 

-0.11841 

*  SIGNIFICANT  AT  ALPHA  =  .05 
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APPENDIX  F-l 


AQCESS  SILAS  B.  HAYS  ARMY  HOSPITAL  FT  ORD,  CA  VS  AQCESS  NAVAL  HOSPITAL 
PENSACOLA,  FL  VS  CHCS  NAVAL  HOSPITAL  CHARLESTON,  SC 


1  TOTAL  SATISFACTION      QUESTION      INDEX 

'PENSACOLA  0.71132  9.02586 

FT  ORD  0.80567  10.47368 

CHARLESTON  0.86558  11.25248 

PENSACOLA  STDEV  1.78610  14.93465 

FT  ORD  STDEV  1.75489  15.73504 

CHARLESTON  STDEV  1.65718  11.73883 

GROUP  FACTORS         A  ABC 

(MID)  (CON) 

PENSACOLA  0.89719  0.05452  0.69885  0.30582 

FT  ORD  1.28363  -0.03955  0.94386  0.36403 

CHARLESTON  1.77288  -0.35891  0.77607  0.52847 

PENSACOLA  STDEV  1.51598  1.72164  1.76039  1.86665 

FT  ORD  STDEV  1.13162  1.77987  1.65140  1.77555 

CHARLESTON  STDEV  1.17591  1.75196  1.51405  1.80255 

INDIVIDUAL  QUESTION  RESPONSES 


QUESTION 
PENSACOLA 
FT  ORD 
CHARLESTON 

1 
1.49713 
1.23684 
1.67822 

2 

-1.04011 
-0.30702 
-0.50000 

3 
0.02012 
0.14035 
0.57426 

4 
0.63793 
0.92998 
0.97525 

5 
0.60632 
0.69299 
1.06436 

STAN  DEV 
PENSACOLA 
FT  ORD 
CHARLESTON 

1 
1.51903 
1.71834 
1.26586 

2 
1.85706 
1.79242 
1.75133 

3 
1.83113 
1.71616 
1.74690 

4 
1.86497 
1.54306 
1.67640 

5 
1.81078 
1.77273 
1.60559 

QUESTION 
PENSACOLA 
FT  ORD 
CHARLESTON 

6 
1.51724 
1.15789 
1.98515 

7 
0.75000 
0.96491 
0.79208 

8 
0.90210 
1.21930 
0.94060 

9 
0.64655 
0.88596 
1.16832 

10 
0.42816 
0.80702 
0.54455 

STAN  DEV 
PENSACOLA 
FT  ORD 
CHARLESTON 

6 
1.56735 
1.79951 
1.04371 

7 
1.78052 
1.66980 
1.49837 

8 
1.69403 
1.63170 
1.43046 

9 
1.75474 
1.70530 
1.55541 

10 
1.71304 
1.56652 
1.18394 

QUESTION 
PENSACOLA 
FT  ORD 
CHARLESTON 

11 
1.36494 
1.45614 
1.65347 

12 
0.14943 
0.44444 
-0.21287 

13 
0.76724 
0.84211 
0.59406 
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STAN  DEV 

11 

12 

13 

PENSACOLA 

1.45492 

1.57589 

1.82113 

FT  ORD 

1.66038 

1.65018 

1.70951 

CHARLESTON 

1.18394 

1.74471 

1.48745 
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APPENDIX  F-2 


CHCS  VS  AQCESS 

TOTAL  SATISFACTION  QUESTION  INDEX 

CHCS  0.86558  11.25248 

AQCESS  0.73459  10.13203 

CHCS  STDEV  1.65718  11.73883 

AQCESS  1.77895  16.68588 


GROUP  FACTORS 

CHCS 
AQCESS 
CHCS   STDEV 
AQCESS  STDEV 


A 
(MID) 
1.77288 
0.89898 
1.17591 
1.57377 


A 
(CON) 
35891 
05844 
75196 


1.73917 


B 

77607 
75931 
51405 
74030 


0 
0 
1 

1 


52847 
32035 
80255 
84525 


INDIVIDUAL  QUESTION  RESPONSES 


QUESTION 

CHCS 

AQCESS 

STAN  DEV 

CHCS 

AQCESS 

QUESTION 

CHCS 

AQCESS 

STAN  DEV 

CHCS 

AQCESS 

QUESTION 

CHCS 

AQCESS 

STAN  DEV 

CHCS 

AQCESS 


1 
1 

1 

.67822 
.43290 

-0 
-0 

2 
.50000 
.10606 

0 
0 

3 
.57426 
.04978 

0 
0 

4 
.97525 
.70996 

1 
0 

5 
.06436 
.62771 

1 
1 

1 
.26586 
.57457 

1 

1 

2 
.75133 
.84215 

1 
1 

3 
.74690 
.80419 

1 
1 

4 
.67640 
71322 

1 
1 

5 
.60559 
.80185 

1 
1 

6 
.98515 
.42857 

0 
0 

7 
.79208 
.80303 

0 
0 

8 
.94060 
98052 

1 
0 

9 
16832 
70563 

0 
0 

10 

.54455 
.52165 

1 
1 

6 
.04371 
.63507 

1 
1 

7 
49837 
.75630 

1 

1 

8 
43046 
68442 

1. 
1. 

9 
55541 
74572 

1 
1 

10 

18394 

68600 

1 
1 

11 

.65347 
.38745 

-0 

0 

12 

.21287 
22294 

0 
0 

13 

59406 

78571 

1 
1 

11 

.18394 
.50874 

1 
1 

12 

74471 

61301 

1 
1 

13 
48745 
79452 
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APPENDIX  F-3 


PENSACOLA  vs  CHARLESTON 


CALCULATED  T-VALUE 


1 

1.0038 

2 

2.37125  * 

3 

2.44113  * 

4 

1.51899 

5 

2.06814  * 

6 

2.68497  * 

7 

0.237516 

8 

1.99893 

9 

2.47011  * 

10 

0.57204 

11 

1.70795 

12 

-1.76089 

13 

-0.843187 

FACTOR 

A(MID) 

-5.044247  * 

A(CON) 

-1.89340 

B 

0.382317 

C 

0.953397 

OVERALL 

1.28452 

*  SIGNIFICANT  AT  ALPHA  =  .05 


164 


APPENDIX  F-4 


FORT  ORD  vs  CHARLESTON 


CALCULATED  T-VALUE 


1 

-1.83458 

2 

0.648655 

3 

-1.49233 

4 

-0.148027 

5 

-1.33802 

6 

-3.63284  * 

7 

0.656472 

8 

1.12314 

9 

-1.03273 

10 

1.22248 

11 

-0.88007 

12 

2.29686  * 

13 

0.95965 

FACTOR 

A(MID) 

-2.55907  * 

A(CON) 

1.09699 

B 

0.61843 

C 

-0.572368 

OVERALL 

-0.353725 

*  SIGNIFICANT  AT  ALPHA  =  .05 
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APPENDIX  G-l 


COMBINED  RESULTS  FOR  CHCS  AND  AQCESS 


GENERAL 

PHYSICIANS 

STDEV 

ANCILLARY 

STDEV 

ADMINISTRATIVE 

STDEV 

GROUP  FACTORS 

PHYSICIAN  SAT 
ANCILLARY  SAT 
ADMIN  SAT 
PHYSICIAN  STDEV 
ANCILLARY  STDEV 
ADMIN  STDEV 
OVERALL  AVG 
OVERALL  STDEV 


AVERAGE 

INDEX 

0.10376 

1.348837 

1.61997 

11.15824 

0.86593 

11.42857 

1.73685 

13.86241 

0.88229 

11.4698 

1.73962 

14.25219 

A 

A 

B 

C 

(MID) 

(CON) 

0.56202 

-0.46899 

0.17907 

-0.48547 

1.64286 

-0.08571 

0.793571 

0.559821 

1.55593 

0.139262 

0.927517 

0.46896 

1.37496 

1.360055 

1.505966 

1.640625 

1.37705 

1.79179 

1.683394 

1.857416 

1.55679 

1.768214 

1.673756 

1.796915 

1.55063 

-0.06777 

0.763441 

0.412634 

1.45332 

1.751599 

1.66848 

1.837235 

INDIVIDUAL  QUESTION  RESPONSES 

QUESTION                 12         3  4         5 

PHYSICIAN               1     -0.72093  -0.27692  0.06153  -0.092 

ANCILLARY             1.63571  -0.29286  0.467857  1.02857  1.0357 

ADMIN                1.53356  -0.01007  0.167785  0.88590  0.8322 


STAN  DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 


12         3         4  5 

1.48637  1.413688  1.476515  1.75381  1.8437 

1.41275  1.882234  1.864816  1.69639  1.6493 

1.53282  1.834576  1.770006  1.70876  1.6945 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


6  7         8         9  10 

1.02326  -0.75581  -0.05814   0.4418  0.2409 

1.73571  0.85  1.017857  0.92143  0.4607 

1.63423  0.922819  1.218121  0.89261  0.6711 


STAN  DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 


6         7  8         9  10 

1.46253  1.630128  1.630439  1.63971  1.4775 

1.36073  1.666583  1.557139  1.71575  1.6664 

1.59816  1.700005  1.56173  1.70919  1.6423 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


11       12        13 

1.37952  -0.36145  0.295181 

1.55714  0.121429  0.717857 

1.5  0.288591  0.932886 
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STAN  DEV  11       12        13 

PHYSICIAN  1.23476  1.517594  1.485951 

ANCILLARY  1.35104  1.671047  1.749194 

ADMIN  1.53538  1.686087  1.749194 
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APPENDIX  G-2 


COMBINED  RESULTS  FOR  CHCS  AND  AQCESS 
PHYSICIANS  VS  THE  REST  OF  THE  SURVEY  GROUP 


GENERAL 

PHYSICIANS 

STDEV 

NON-PHYSICIANS 

STDEV 

GROUP  FACTORS 

PHYSICIAN  SAT 

PHYSICIAN  STDEV 

NON-PHYSICIANS 

NON-PHYSICIANS  STDEV   1.4731  1.783479  1.679766  1.827038 


AVERAGE 
0.10376 
1.48637 
0.87437 
1.73957 

INDEX 

1.348837 

11.15824 

11.44983 

14.06473 

A 

(MID) 

0.56202 

1.37496 

1.59804 

A 
(CON) 
-0.46899 
1.360055 
0.295848 

B 

0.17907 

1.505966 

0.86263 

C 

-0.48547 
1.640625 
0.512976 

QUESTION 

PHYSICIAN 

NON-PHYSICIANS 


12  3  4  5 

1  -0.72093    -0.27692   0.061538    -0.09231 

1.58304    -0.27682   0.313149    0.955017    0.930796 


STAN  DEV 

PHYSICIAN 

NON-PHYSICIANS 


12         3         4         5 
1.48637  1.413688  1.476515  1.753811  1.843711 
1.47676  1.863184  1.822732  1.704274  1.675816 


QUESTION 

PHYSICIAN 

NON-PHYSICIANS 


6         7         8         9        10 
1.02326  -0.75581  -0.05814   0.44186  0.240964 
1.68339  -0.14706  1.121107  0.906574  0.569204 


STAN  DEV 

PHYSICIAN 

NON-PHYSICIANS 


6         7         8         9        10 
1.46253  1.630128  1.630439  1.639705  1.477464 
1.48875   1.68429  1.562716  1.712439   1.65739 


QUESTION 

PHYSICIAN 

NON-PHYSICIANS 


11       12        13 
1.37952  -0.36145  0.295181 
1.52768  0.205882   0.82872 


STAN  DEV 

PHYSICIAN 

NON-PHYSICIANS 


11       12        13 
1.23476  1.517594  1.485951 
1.44929  1.681623  1.730074 
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APPENDIX  G-3 


PHYSICIANS  vs  THE  REST  OF  THE  SURVEY  GROUP 


CALCULATED  T-VALUE 


1 

2.40721* 

2 

1.52642 

3 

1.98308* 

4 

3.23135* 

5 

3.65143* 

6 

2.78686* 

7 

2.20323* 

8 

4.62484* 

9 

1.68446 

10 

1.19979 

11 

0.64683 

12 

2.10863* 

13 

1.91295 

FACTOR 

A(MID) 

4.38365* 

A(CON) 

2.69228* 

B 

2.49046* 

C 

3.41617* 

OVERALL        4.52104* 
*  SIGNIFICANT  AT  ALPHA 


=  .05 
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APPENDIX  G-4 


COMBINED  RESULTS  FOR  CHCS  AND  AQCESS 
ANCILLARY  VS  THE  REST  OF  THE  SURVEY  GROUP 


GENERAL 

ANCILLARY 

ANCILLARY  STDEV 

NON-ANCILLARY 

NON- ANCILLARY  STDEV 

GROUP  FACTORS 

ANCILLARY  SAT 
ANCILLARY  STDEV 
NON- ANCILLARY 
NON- ANCILLARY  STDEV 


AVERAGE  INDEX 
0.86621  11.42857 
1.73663  13.86241 
0.70753  9.203125 
1.74605  14.25906 

A         A         B         C 
(MID)     (CON) 

1.64286  -0.08571  0.793571  0.560714 

1.37705   1.79179  1.683394  1.856906 

1.43837  -0.05599  0.727909  0.254557 

1.53366   1.72398  1.667132  1.807729 


QUESTION 
ANCILLARY 
NON -ANCILLARY 

STAN  DEV 
ANCILLARY 
NON- ANCILLARY 

QUESTION 
ANCILLARY 
NON -ANCILLARY 

STAN  DEV 
ANCILLARY 
NON- ANCILLARY 

QUESTION 
ANCILLARY 
NON -ANCILLARY 

STAN  DEV 

ANCILLARY 

NON-ANCILLARY 


12         3         4  5 

1.63571  -0.29286  0.467857  1.028571  1.039286 

1.41406  -0.17708  0.020833  0.617188  0.557292 

12        3         4  5 

1.41275  1.882234  1.864816  1.696395  1.64595 

1.53871  1.776553  1.730421  1.790256  1.80331 

6  7        8         9  10 

1.73571  0.85  1.017857  0.921429  0.460714 

1.4974  0.763021  0.932292  0.791667  0.578125 

6  7        8         9  10 

1.36073  1.666583  1.557139  1.715758  1.666442 

1.58935  1.746181  1.647472  1.690455  1.608616 

11  12        13 

1.55714  0.121429  0.717857 

1.40365  0.065104  0.734375 

11  12        13 

1.35104  1.671047  1.749194 

1.4688  1.660958  1.677754 
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APPENDIX  G-5 


ANCILLARY  vs  THE  REST  OF  THE  SURVEY  GROUP 


CALCULATED 

T-VALUE 

1 

-1.45036 

2 

0.61534 

3 

-2.40663* 

4 

-2.18425* 

5 

-2.55840* 

6 

-1.44183 

7 

-0.49104 

8 

-0.46473 

9 

-0.71419 

10 

0.68605 

11 

-1.06003 

12 

-0.30922 

13 

0.10936 

FACTOR 

A(MID) 

-1.28482 

A(CON) 

0.11483 

B 

-0.33643 

C 

-1.56279 

OVERALL 

-1. 

48159 


*  SIGNIFICANT  AT  ALPHA  =  .05 
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APPENDIX  G-6 


COMBINED  RESULTS  FOR  CHCS  AND  AQCESS 
ADMINISTRATIVE  VS  THE  REST  OF  THE  SURVEY  GROUP 


GENERAL 

ADMINISTRATIVE 
ADMIN  STDEV 
NON-ADMIN 
NON-ADMIN  STDEV 

GROUP  FACTORS 

ADMIN  SAT 
ADMIN  STDEV 
NON-ADMIN  SAT 
NON-ADMIN  STDEV 

QUESTION 

ADMIN 

NON-ADMIN 

STAN  DEV 

ADMIN 

NON-ADMIN 

QUESTION 

ADMIN 

NON-ADMIN 

STAN  DEV 

ADMIN 

NON-ADMIN 

QUESTION 

ADMIN 

NON-ADMIN 

STAN  DEV 

ADMIN 

NON-ADMIN 


AVERAGE  INDEX 
0.88178   11.4698 


1.74002 
0.68705 
1.74022 


14.2522 

9.06011 
14.25906 


B 


A         A 
(MID)     (CON) 

1.55593  0.137584  0.927517  0.468121 

1.55679  1.768819  1.673756  1.797367 

0.64918  0.314891 

1.66395  1.862015 


1.52302  -0.23634 
1.40079  1.721812 


12  3         4         5 

1.53356  -0.01007  0.167785  0.885906  0.828859 

1.48634  -0.40164  0.243169  0.713115  0.704918 

12  3         4         5 

1.53282  1.834576  1.770006  1.708762  1.697088 

1.45556  1.794009  1.826758  1.802304  1.798703 

6  7  8         9        10 

1.63423  0.922819  1.218121  0.892617  0.671141 

1.56831  0.699454  0.765027  0.808743  0.412568 

6  7  8         9        10 

1.59816  1.700005  1.56173  1.709197  1.642322 

1.41787  1.658675  1.621064  1.695874  1.618458 

11  12  13 

1.5  0.285235  0.932886 

1.44262  -0.07104  0.560109 

11  12  13 

1.53538  1.687653  1.749194 

1.32266  1.629772  1.692298 
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APPENDIX  G-7 


ADMINISTRATIVE  vs  THE  REST  OF  THE  SURVEY  GROUP 


CALCULATED  T-VALUE 


1 

0.00000 

2 

-2.01622  * 

3 

-0.36259 

4 

-0.95500 

5 

-0.69139 

6 

-0.43885 

7 

-1.09948 

8 

-2.65698 

9 

-0.52758 

10 

-1.57692 

11 

-0.39612 

12 

-1.97450  * 

13 

-2.00788  * 

FACTOR 

A(MID) 

-0.25394 

A(CON) 

-1.82145 

B 

-1.68437 

C 

-1.01902 

OVERALL 

-1.59613 

*  SIGNIFICANT  AT  ALPHA  =  .05 
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